Monitoring and evaluation platform for HEARTS in the Americas: improving population-based hypertension control programs in primary health care.

HEARTS in the Americas is the Pan American Health Organization flagship program to accelerate the reduction of the cardiovascular disease (CVD) burden by improving hypertension control and CVD secondary prevention in primary health care. A monitoring and evaluation (M&E) platform is needed for program implementation, benchmarking, and informing policy-makers. This paper describes the conceptual bases of the HEARTS M&E platform including software design principles, contextualization of data collection modules, data structure, reporting, and visualization. The District Health Information Software 2 (DHIS2) web-based platform was chosen to implement aggregate data entry of CVD outcome, process, and structural risk factor indicators. In addition, PowerBI was chosen for data visualization and dashboarding for the analysis of performance and trends above the health care facility level. The development of this new information platform was focused on primary health care facility data entry, timely data reporting, visualizations, and ultimately active use of data to drive decision-making for equitable program implementation and improved quality of care. Additionally, lessons learnt and programmatic considerations were assessed through the experience of the M&E software development. Building political will and support is essential to developing and deploying a flexible platform in multiple countries which is contextually specific to the needs of various stakeholders and levels of the health care system. The HEARTS M&E platform supports program implementation and reveals structural and managerial limitations and care gaps. The HEARTS M&E platform will be central to monitoring and driving further population-level improvements in CVD and other noncommunicable disease-related health.

Cardiovascular diseases (CVD) cause more deaths in the Americas than any other disease, accounting for close to one third of total deaths in 2017 (1). In response, HEARTS in the Americas (HEARTS) was launched in 2017 as a multicountry program, led by the ministries of health with technical assistance from the Pan American Health Organization (PAHO). Integrating HEARTS into existing health delivery services, particularly at the primary health care level, is critical to encouraging the adoption of global best practices in preventing and controlling CVD and other noncommunicable diseases. HEARTS is currently implemented in 23 Latin American and Caribbean countries, including more than 1300 primary health care facilities (2, 3).

The lack of a quality improvement system, including insufficient data collection and timely reporting mechanisms to identify areas for potential improvement, particularly at the primary health care level (2, 3), are significant challenges impeding a rapid scaling up of HEARTS. Therefore, developing a standardized, feasible, and responsive monitoring and evaluation (M&E) platform is vital for implementing HEARTS (4). In addition, reporting, learning from peer-to-peer communication, and strengthening best practices to inform policy, quality, and system monitoring are paramount to further CVD health improvements (5). This effort is aligned with PAHO’s resolution on the digital transformation of the health sector in the Region of the Americas (6).

This paper describes the conceptual bases of a web-based HEARTS M&E platform, developed to fill these critical M&E gaps, as informed through M&E system design principles (7). Additionally, lessons learnt and programmatic considerations are assessed through the experience of the M&E software development. This new HEARTS M&E platform will focus on data entry from primary health care facilities, timely data reporting, and ultimately active use of data to drive decision-making, equitable program implementation, and improved quality of care.

KEY DESIGN PRINCIPLES FOR THE M&E SYSTEM

The HEARTS digital M&E platform focuses on data entry and uses within primary health care facilities. This platform is designed based on three fundamental principles: advocacy, accountability, and action.

Advocacy

Monitoring data on allocation of resources, such as staff and materials, organizational practices, processes, and outcomes, is central for assessing and contextualizing structural and managerial factors that hinder and improve the program and patient health outcomes. Additionally, reporting data by sex, age, CVD risk strata, race and ethnicity, and community socioeconomic status allows health facilities to monitor inequitable outcomes. This information is critical to successful HEARTS implementation and can drive policy changes.

Accountability

A data platform enables cross-facility and cross-country comparisons of current progress improvements, implementation maturity, and performance level. In addition, access to data ensures up-to-date and timely reports on performance and identification of the most relevant drivers and bottlenecks contextualized to population dynamics and needs. Objective and standardized data feedback is more likely to create a culture of accountability and positive change by informing and setting attainable goals and consistently monitoring outcomes.

Action

Establishing communities of practice (on sharing best practices and creating new knowledge to advance professional practice) and strategic inter- and intra-country partnerships is the main focus for developing a solid feedback and quality improvement system (5). Linking structural- and community-level factors, facility performance indicators, and population-level health outcomes is important for contextualizing results. Considerations of facility context through analysis of similar policies, demographics, and resources are necessary for appropriately comparing outcomes, progress, and goal setting.

CONTEXTUALIZATION OF THE DATA COLLECTION

The M&E system collects structural health care indicators, and demographic and community indicators, processes, and outcomes to identify relevant weak and strong points in HEARTS implementation, program maturity, and performance. Analyzing these indicators contextualized through health facility characteristics and needs is essential to understanding disparities in magnitude and progress. Clusters and characteristics associated with low performance and progress will be targeted to drive improvements. An information system and data flow allow for more readily available, actionable data to achieve equity in hypertension control, including control of hypertension among people with a high risk of CVD and diabetes. Data collection tools, fully aligned with the WHO HEARTS technical package on systems for monitoring, were designed by PAHO with input from ministries of health of implementing countries, the US Centers for Disease Control and Prevention, and Resolve To Save Lives.

Modules

Data collection was organized into content-specific modules informed by the socioecological model (8), health care facility needs, and progress in achieving equitable programmatic outcomes (Figure 1). In addition, process and outcome modules are based on key drivers for hypertension control monitored through maturity and performance indexes, which is an innovative approach introduced by HEARTS (9). All data are collected as aggregate monthly counts per health facility. The modules were organized as follows.

FIGURE 1.

High-level structure of HEARTS monitoring and evaluation platform

Geolocalization and typology module. Geographic and health care facility characteristics affecting treatment options and care access.

Geographic location

Facility setting type (rural/urban)

Level of complexity based on the type of health care services provided.

Structure module. Health care facility resources and personnel indicators affecting the ability to meet patient needs and appropriate standards of care.

Number and type of medical staff

Availability and location of pharmacies, laboratories, and medications

Availability of automated validated blood pressure devices

Use and type of electronic health information systems

Use of multi-month prescriptions, telemedicine, and remote blood pressure monitoring.

Demographics module. Demographic characteristics of the population served within the health care facility catchment area to help understand patient needs and outcomes.

Sex and age distribution

Socioeconomic status of the surrounding community where the health center is located

Race and ethnicity.

Process module. Periodic process indicators related to health facility implementation maturity, including continuous quality improvement measures.

Blood pressure certification of staff, repeated blood pressure measurements, and validated automated blood pressure measurement devices

Assessment of CVD risk

Use of standardized treatment protocol and fixed-dose combination medicines for hypertension

Initiation and intensification of treatment

Continuity of care and follow-up

Team-based care and the use of task sharing

Medication refill frequency

System for performance evaluation with feedback.

Outcomes module. Patient-focused M&E outcome indicators core to HEARTS implementation, progress monitoring, and reporting. Indicators are reported monthly by age, sex, and CVD risk strata (general population, high-risk groups, and people with diabetes).

Estimated population with hypertension within the catchment area

Individuals with hypertension in treatment at the health facility

Individuals with hypertension with scheduled visits

Individuals with hypertension with completed clinical visits at the health facility (retention)

Population with hypertension with improved blood pressure control.

A detailed list of definitions of HEARTS scorecard indicators, both maturity and performance, has been published elsewhere (9).

DATA ENTRY

In order to rapidly assess and report on outcomes, modules were built within DHIS-2. DHIS-2 is open-source software used to monitor, evaluate, and report individual and aggregate data in resource-constrained settings (10). Considerations and requirements for using the system included implementing data collection entirely online without installing software through a web-based interface, secure access to data and data entry, and quality-focused data entry with validation controls. Privacy and confidentiality risks were reduced by collecting aggregate indicators. In addition, consideration of integrity, security, and confidentiality was paramount in moving from data aggregation and integration of Excel sheets to secure online servers.

CENTRAL DATA REPOSITORY AND DATA WAREHOUSE

DHIS-2 was chosen as the centralized data repository for all aggregate indicators using the DHIS-2 standard data model, allowing for immediate reporting and visualizations at the health facility level. From module data within DHIS2, additional transformations, data cleaning, and integration of other data sources into a dimensional model enable more dynamic features to navigate data trough tools such as filters, ad hoc tables for an in-depth analysis of processes, and outcomes within Microsoft PowerBI. PowerBI is a flexible and powerful data visualization and dashboard solution, which allows integration of various data endpoints, including DHIS2. The data warehouse contains verified, carefully curated, and cleaned data to derive national and international reports and visualizations.

DATA VISUALIZATIONS AND REPORTING

Immediate health facility-focused trends and current outcomes were developed within DHIS2 for instantaneous reporting once data are entered. The reports were mocked up in standardized dashboards that health centers can access to enable collaborative analysis. This process ensures the identification of the most prevalent, multilevel areas of improvement affecting HEARTS implementation within the context of a single health care facility. In addition, this enables staff at a health care facility to quickly assess current and past performance indicators and focus on gaps resulting in inequitable outcomes. Rapid reporting and feedback cycles allow for prompt evaluation of changes in workflows and interventions to enhance performance improvement.

A dashboard and data visualization solution was developed in Microsoft PowerBI for more comprehensive analysis and integration above the site level.

The dashboard (Figure 2) was designed to display the most relevant outcome indicators on the left side of the screen while also keeping any filters to the right side of the screen, which are necessary for manipulating data and describing trends within specific groups. Dedicated graphs display data trends for outcomes, such as coverage and control over time, with the ability to compare across national trends. In addition, demographics and geographic location are displayed and can be filtered, which is important for contextualizing the population and community being serviced. Outcome and process indicators (and constituent components) are also displayed for each health facility, with the ability to compare current standing across multiple facilities. Hypertension treatment cascade, coverage, retention, and control stratified by CVD risk are also displayed to compare progress and gaps.

FIGURE 2.

Example of national above-site focused dashboard design

The HEARTS M&E platform is in line with the principles of interoperability and scalability. The flexibility allowed by aggregate data collection improves scalability to multiple health systems as the program and data needs grow. DHIS2 modules can be extended and adjusted to meet new programmatic considerations or changes in indicators. The need to integrate tools that accurately aggregate finer individual-level CVD data for reporting into DHIS2 will be evaluated and considered.

LESSONS LEARNT AND PROGRAMMATIC CONSIDERATIONS

This HEARTS M&E platform, with an already established data collection and operationalization methodology (11), was co-created by a multidisciplinary group that included data scientists, epidemiologists, public health physicians, quality improvement specialists, informaticians, computer engineers, subject matter experts, and PAHO technical advisers. The platform is hosted by PAHO.

Since the primary health care center is the focus of HEARTS, there was a need to establish a well designed, incentivizing system to promote the use of data for facility-level improvement of process and patient outcomes and services beyond just meeting reporting requirements. Thus, data standardization and innovation of data utilization (12), one of the HEARTS technical pillars, served as the backbone for implementation. Recognizing that political buy-in is as critical as the technology that enables a learning health care system and sustainability, the authors engaged multiple stakeholders and end-users within various ministries of health for design input and field testing. As a result, the dashboard tools and indicators were iteratively developed and operationalized with the flexibility to accommodate health and reporting system needs and contexts, ranging from advanced electronic medical record systems to paper-based patient tracking systems.

Holistic analysis of structural factors through data collection modules enabled contextualization of improvements and gaps in achieving the primary outcomes of HEARTS coverage and control indicators. In addition, given the resource constraints for data reporting at the primary health care facility level, the authors recommend collecting secondary contextual factors, in a standardized manner, at higher levels of the health system.

The refocus of data use at the primary health care facility level is central to enhancing the data-driven culture of quality improvement. Increasing data use re-establishes the connection between patient population needs and contextualized outcomes. Under-resourced health facilities with low programmatic performance and poor health outcomes are encouraged to use their empirical data to advocate for resources and political support to achieve equitable outcomes across various demographics and geographic areas in the Americas.

Further efforts on capacity-building and investments will be required to continually improve the operation of this platform at the primary health care facilities. Implementation research will help us to understand and improve its operation. Establishing communities of practice is necessary to identify similar contexts, prioritize efficacious interventions, and promote best practices across implementing areas.

The HEARTS M&E platform will not solve existing challenges around data collection and analysis in all countries but it will set the standard of best practices around M&E in hypertension, CVD and noncommunicable disease programs. Indeed, this platform can support program implementation, reveal structural and managerial limitations and care gaps, and lead to favorable changes at different levels of the health system.

The HEARTS M&E platform has an important limitation. Indeed, it is a well known issue for aggregate data collection systems, namely, the system is unable to track individual outcomes. Since different health systems are heterogenous in their ability to track individual patient data – some still rely on paper forms – we have opted for maximum flexibility at the expense of generating patient-level insight. The practical implication is that when we sample and report on these indicators, it is not guaranteed that we are reporting on the same individuals. We try to limit the potential impact of this through collecting additional indicators on retention and reducing our reporting frequencies from quarterly to monthly. The retention indicator will track how often patients are returning for their follow-up appointments, which can be used to assess how likely these patients are being reported on longitudinally. By increasing reporting frequency, changes can be assessed more readily and the impact of changing populations within these indicators can be controlled (during analysis and reporting).

Moreover, beyond the technical considerations inherent in a platform such as the HEARTS M&E platform, we can anticipate multiple challenges in implementation. For instance, the lack of an institutionalized culture based on continuous management of quality improvement, lack of incentives for health care staff and facilities, and resistance to or unfavorable perception of evaluation or clinical audit are a few potential threats to our proposed approach. Nonetheless, by exemplifying data use at lower levels of the health system, we can increase investment in these processes, in data collection efforts, and the use of data and visualizations.

Finally, the precariousness of health information systems in many places, including limited access to the Internet and understaffing due to low priority and investment, has prevented these systems being developed at the same speed as information technologies and communications. Another barrier to implementing this platform is the absence of policies obliging health authorities to share health outcomes and population management data with the communities they serve. Although not exclusive to the primary health care level, these deficiencies negatively affect individual care and CVD outcomes.

CONCLUSION

HEARTS is by nature a data-driven program. The concurrent development and implementation of data entry tools, visualizations, and dashboards exemplify a commitment to supporting data use at the health care facility level to improve accurate and inclusive monitoring and drive improvements in health outcomes. In addition, these tools allow health care teams and communities to remain informed and engaged and hold the leadership at different levels of the system accountable for their management. The implementation of this system goes beyond obtaining data and improving the leading indicators of the program. Instead, its ambition is to reveal unjustified inequities and generate significant investment to address the most disadvantaged areas. As a result, it can promote efficacious interventions to reduce the gap in access to and quality of care and produce better health outcomes.

Disclaimer.

The authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the Revista Panamericana de Salud Pública / Pan American Journal of Public Health and/or those of the Pan American Health Organization, nor the Centers for Disease Control and Prevention, the Department of Health and Human Services, the Task Force for Global Health, Inc., or TEPHINET.