Bridging the impactibility gap in population health management: a systematic review.

OBJECTIVES: Assess whether impactibility modelling is being used to refine risk stratification for preventive health interventions.
DESIGN: Systematic review.
SETTING: Primary and secondary healthcare populations. PAPERS: Articles published from 2010 to 2020 on the use or implementation of impactibility modelling in population health management, reported with the terms 'intervenability', 'amenability', and 'propensity to succeed' (PTS) and associated with the themes 'care sensitivity', 'characteristic responders', 'needs gap', 'case finding', 'patient selection' and 'risk stratification'.
INTERVENTIONS: Qualitative synthesis to identify themes for approaches to impactibility modelling.
RESULTS: Of 1244 records identified, 20 were eligible for inclusion. Identified themes were 'health conditions amenable to care' (n=6), 'PTS modelling' (n=8) and 'comparison or combination with clinical judgement' (n=6). For the theme 'health conditions amenable to care', changes in practice did not reduce admissions, particularly for ambulatory care sensitive conditions, and sometimes increased them, with implementation noted as a possible issue. For 'PTS modelling', high costs and needs did not necessarily equate to high impactibility and targeting a larger number of individuals with disorders associated with lower costs had more potential. PTS modelling seemed to improve accuracy in care planning, estimation of cost savings, engagement and/or care quality. The 'comparison or combination with clinical judgement' theme suggested that models can reach reasonable to good discriminatory power to detect impactable patients. For instance, a model used to identify patients appropriate for proactive multimorbid care management showed good concordance with physicians (c-statistic 0.75). Another model employing electronic health record scores reached 65% concordance with nurse and physician decisions when referring elderly hospitalised patients to a readmission prevention programme. However, healthcare professionals consider much wider information that might improve or impede the likelihood of treatment impact, suggesting that complementary use of models might be optimum.
CONCLUSIONS: The efficiency and equity of targeted preventive care guided by risk stratification could be augmented and personalised by impactibility modelling. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Comparing data was difficult due to widespread inconsistency in terminology.

The quality of the articles included in this review was not graded.

This is a growing area of interest and few studies are available for assessment.

We were as inclusive as possible with types of article, including abstracts and grey literature.

To make the findings most applicable to population health management, we excluded studies of specific diseases.

Introduction

The triple aim is targeted towards improving the individual experience of care, improving the health of populations, and reducing the per capita costs of care,1 and has become a popular healthcare objective. Risk stratification is one type of population health management (PHM) tool used by health system managers to achieve the triple aim2–5 and identifies groups that are at high risk of poor outcomes so that they can be offered preventive care aimed at lowering this risk. For instance, care in accident and emergency has high costs and a cohort of patients experience frequent attendances, making this cohort a potential target for increased preventive spending. However, within this high-risk cohort, some individuals may be labelled as being ‘beyond help’ because they their attendance is perceived by clinicians to be non-preventable (eg, because of age, sex or chronic conditions, including alcohol or drug abuse).2 3 For these individuals, preventive care interventions will have little or no effect and they will continue to be at risk of so-called triple-fail events (in this case accident and emergency attendances), which are harmful, costly and result in poor patient satisfaction.4 6–9

While risk stratification models may accurately predict which individuals are at risk of future adverse health outcomes, such as readmission or 1-year mortality,2–5 their use has not consistently led to improvements in health outcomes across the population.10 Calculating and understanding the probability of a particular outcome for an individual may not be enough for healthcare professionals to intervene in the most efficient way to delay or prevent that outcome or divert the course of a disease. This often needs to be supported by additional information to determine the most accurate or appropriate model.11 Furthermore, as many risk stratification models predict future adverse health outcomes through current or previous healthcare activity and use a limited number of variables,12–15 they may miss out on valuable additional information that could better direct resources to patients amenable to benefit.9 16 Lewis6 defined a different type of model—impactibility models—that are aimed at identifying the subset of at-risk patients for whom preventive care is expected to be successful.

Lewis6 found that impactibility was being assessed by many healthcare systems for PHM, reflecting a growing recognition that not all high-risk patients will benefit from preventive care. He described the ideal impactibility model as one that ‘would use information about the differential effects of a specific preventive intervention offered at random to patients and controls, so as to identify the characteristics of the ‘perfect patient’ for that preventive programme’. However, suitable data are rarely available in real-world records. Instead, he found that models were being formulated in three main classes: ‘(1) giving priority to patients with diseases that are particularly amenable to preventive care; (2) excluding patients who are least likely to respond to preventive care or (3) identifying the form of preventive care best matched to each patient’s characteristics’. While such impactibility models have considerable potential to improve the efficiency of preventive care delivery, certain approaches could increase health inequalities if used indiscriminately without catering to individual needs.6 The aim of this current study was to describe broadly how and in what contexts impactibility modelling has been implemented or assessed in PHM since 2010. We defined impactibility as the identification of patients most likely to respond to care based not only quantitative but also on qualitative factors, and whose treatment would maximise the likelihood of achieving the triple aim. It was beyond the scope of this review to consider how impactibility modelling might affect management of individual diseases, heterogeneity in treatment effects and different types of health programmes.

Methods

A systematic literature review was carried out to identify all papers published between January 2010 and May 2020. The Ovid search platform was used to search four relevant databases: Embase Classic and Embase, Global Health, Healthcare Management Information Consortium and Ovid MEDLINE. Additional searches for grey literature were performed in OpenGrey.

Search strategies were built iteratively, with relevant keywords and subject headings for each database added based on initial reviews of relevant publications. The final set of search terms (see online supplemental information pp 1‒28) included alternative spellings of impactibility and synonyms, including ‘intervenability’, ‘amenability’ and ‘propensity to succeed’. We also included words associated with the following themes: ‘care sensitivity’, ‘characteristic responders’, ‘needs gap’, ‘case finding’, ‘patient selection’ and ‘risk stratification’. Where relevant, these search terms were linked with the Boolean ‘and’ operator to synonyms for ‘predictive model’, ‘population health’ or ‘preventive healthcare’. No additional restrictions were applied in terms of language, date or status of publication.

Database search results were exported to the systematic review software Covidence. Two reviewers (AO and SS) independently screened titles and abstracts for relevance and reviewed the full texts that specifically referenced analyses of amenability, impactibility, and propensity to succeed (PTS) in relation to future events. Papers that concerned youth offending, aimed to increase screening detection rates, and looked only at identifying individuals at high risk of a specific disease or health event were excluded. Full inclusion and exclusion criteria are shown in online supplemental information (pp 29‒31). To achieve the widest possible overview of work in this emerging field, studies were not excluded based on assessment of methodological quality. Conflicts were discussed with a third reviewer (WS) at each review stage. A pragmatic forward citation search was subsequently conducted using PubMed for all articles included in the initial review round. These were added to Covidence, and the screening process was repeated. A targeted Google search (see online supplemental information pp 32) was conducted to identify any additional publications containing the term ‘impactibility’.

Data extraction was performed by SS, HH and WS. For studies describing impactibility models, information about country of implementation, data sources, population studied, intervention and any reported outcome measures were extracted into a data table. Qualitative synthesis was performed to assess themes and to group papers by approach to impactibility modelling.17 Outcome measures, where reported, were not comparable across studies so meta-analysis was not considered to be appropriate.

Results

Of 1244 records initially identified, 179 full-text items were assessed for eligibility after removal of duplicates and initial exclusion based on title and abstract. Of these, 81 were found to be ineligible and 78 were commentaries. Thus, 20 studies related to the development, application or validation of impactibility models for use in PHM and were included in the review (figure 1).

Figure 1

PRISMA diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

In the qualitative synthesis, we grouped papers under three themes representing different approaches to assessing impactibility: health conditions amenable to preventive care (n=6); PTS (n=8) and comparison or combination with clinical judgement (n=6; see online supplemental information pp 33‒43).

Health conditions amenable to preventive care

Several studies inferred participants’ potential to benefit from preventive care if it is targeted after they have received a diagnosis of a specific health condition18–21 or if they have a multimorbid cluster of health conditions.22 23 Many of these studies specifically targeted people with ambulatory care sensitive conditions (ACSCs), including chronic obstructive pulmonary disease, chronic heart failure and diabetes, for which evidence suggests that optimal management in the community should not result in unplanned hospital admission.10 16 24 25 Preventive interventions (eg, case management) that were targeted based on the presence of one or more ACSC did not consistently lead to reductions in hospital admissions or secondary care costs, and indeed, in some cases led to increases in emergency hospital admissions.18–22 However, the success of these impactibility strategies may be hindered by ineffective implementation. In one of these studies, for example, the authors indicated that the targeted intervention was not effectively integrated into primary care practice during the observation period.21

Propensity to succeed

PTS modelling is an analytical approach to identify traits associated with improved engagement with or outcomes from particular preventive health intervention(s)—outcomes such as cost or care quality.26–32 Of the eight studies identified that used this approach, three used PTS modelling in relation to specific case management interventions.30–32 One model was developed explicitly for ‘low-risk’ participants to assess who would be most likely to benefit from a digital health platform.28

In these studies, PTS regression analyses were performed using various sociodemographic factors,26–28 30–32 health status (eg, presence of chronic conditions, prescription data, prior health resource utilisation and various health risk scores)26–28 30–32 or previous programme engagement metrics.28 One study found that high costs and high needs did not equate to high impactibility, as only small proportions of people with diseases that would be expected to have high burden had scores indicating high impactibility. The authors suggested that targeting a larger number of individuals with disorders associated with lower costs could improve impact substantially and that better predictors of impactibility might be medication adherence and historical healthcare resource utilisation that was unexplained by disease burden.31

Five of the identified studies reported the statistical validity of PTS models for projecting cost savings, improved engagement and/or care quality improvements;26–28 30 32 however, prospective or comparative outcome data on the use of these models in real-world situations were extremely limited in the literature. Two studies reported improved engagement (defined as enrolment of contacted participants) with case management interventions after implementation of a PTS model: Ozminkowski et al32 reported an 11% increase in programme enrolment in the 9 months after implementation of a PTS model, compared with the 3 months prior. Hommer et al29 likewise reported increased enrolment in a depression management programme but did not quantify the change.

Hsueh et al33 evaluated the Behavioural Response Inference Framework (BRIeF), a machine learning impactibility model derived from a large observational dataset of care management records from a private healthcare network. They tested the ability of the model to predict individual-level behavioural responses to multiple interventions used in care planning. Input data included participants’ personalised goal attainment history across 16 goals set in a programme to reduce hospital readmissions after discharge for acute care. They covered a wide spectrum of care needs (eg, tobacco cessation, knowledge of healthy eating, medication adherence, actions to resolve care gaps, and fall prevention) and were categorised as ‘met’, ‘abandoned’, ‘not met’ or ‘open’. Data on goal attainment were extracted for 131 different care coordination activities in the categories referral, education, coordination, screening, coaching or other tasks, that were classified as met or otherwise. The BRIeF model was applied to assess behavioural responses at the individual patient and population levels. Covariates used in the model were demographic information (eg, age and gender), care programme context (eg, programme experience and days in the programme) and the interactions between care managers and patients (eg, the day of making the recorded call). The authors described the results of the model as ‘promising’, with the individual-level care planning strategy showing the greatest accuracy in terms of correct intervention recommendations, which outperformed a population-level care planning approach where the one-size-fits-all approach reduces precision.

Comparison or combination with clinical judgement

We identified six impactibility models that—either formally or informally—incorporated a healthcare provider’s opinion of whether an individual patient was likely to benefit from a particular preventive health intervention.16 34–39 In one study, clinical judgement was applied as a final (filtering) step to estimate how care management would impact patients after they had undergone risk stratification by a predictive analytical tool.40 A predictive tool calculated a risk score for emergency department visits in the next 12 months based on 19 variables. Physicians then added information on medical and social factors that could alter the impact of care management. This combined improved identification of higher-risk patients, reflected by an increase in the average risk score for patients enrolled in care management from 33.4% to 40.4%.

Cohen et al41 designed a predictive model to identify patients who would benefit from proactive multimorbid care management based on inclusion and exclusion criteria refined from a physician survey of 375 cases and on risk of future high costs based on data extracted from a health services database. Recommended reasons for exclusion due to risk of future high costs were active cancer, schizophrenia, dialysis, residence in nursing homes or long-term care facilities and age 95 years or older. The model was used to assess 5341 high-risk patients. The discriminatory power of the model before and after clinical exclusions was c-statistic 0.80 and 0.75, respectively. Age, number of chronic conditions and healthcare utilisation were associated with high-risk of high-cost care. The authors concluded that the model had acceptable discriminatory power for identifying who would benefit from proactive care management even after the highest-risk patients were excluded.

HCPs consider a range of factors when assessing an individual’s suitability for a preventive care intervention. These include perceived hospitalisation risk; feelings of sympathy or aversion towards the patient and a judgement of the patient’s willingness and ability to participate in the intervention.16 38 HCPs also reported excluding patients from preventive healthcare interventions because of language barriers.16

Flaks-Manov et al42 investigated whether risk scores for 30-day readmission from an electronic health records model were aligned with nurses’ and physicians’ perceived impactibility of a readmission prevention programme for hospitalised patients aged 65 years or older. The clinical and model decisions for 435 patients were concordant in 65% of cases. Among the remaining 35%, 19% with high model scores were not referred by healthcare professionals and 16% with low model scores were referred. Decision-tree analysis indicated that as well as high models scores, eligibility for a nursing home, having a condition not under control, need for social-services support and need for special equipment at home were statistically associated with referral. The authors concluded that better understanding is needed of whether combining perceptions and modelling could improve selection of patients.

Freund et al43 assessed areas in which impactibility modelling might be helpful. They invited 12 primary-care physicians in ten practices to review records for 104 hospitalisations in 81 patients who had ACSCs and rate whether they felt each was avoidable. The doctors deemed 43 (41%) hospitalisations to be avoidable. Reasons fell into five main categories: system related (eg, unavailability of ambulatory services), physician related (eg, suboptimum monitoring), medical (eg, medication side effects), patient related (eg, delayed help-seeking) and social (eg, lack of social support). Further reasons were after-hours referral required in the absence of the treating physician, not using ambulatory services, patients’ fears, cultural background, language skills, medication errors, non-adherence to medication and overprotective caregivers. In discussing implications for clinical practice and policy, it was suggested that the risk stratification modelling could be enhanced by considering patients’ social situation, medication adherence and self-management capabilities and sharing responsibility across sectors.

Discussion

Key findings

As health systems turn to data-led approaches to deliver the triple aim of improving individuals’ experience of care and the health of populations while reducing per capita care costs,1 many are finding that allocating resources based on risk stratification alone is suboptimal. Targeting patients for preventive care based only on health conditions amenable to preventive care does not necessarily lead to reductions in resource use and might even increase it,10 and recognition is growing that these goals will only be met if treatment is successful. This is the impactibility gap. Thus, rather than trying to identify patients by negative outcomes (eg, high cost of care, most severe disease), the importance of identifying patients in whom care options will be most effective is being realised. The evidence reviewed shows varying attempts to make prediction tools more impactful and effective by considering the probability of success of interventions. PTS modelling showed some of the most promising results when broader information, such as sociodemographic factors, medication adherence or previous programme engagement, was included. The accuracy of predicting behavioural responses seems to be greatest at the individual level, but more data on real-world outcomes are needed, as implementation could affect PHM potential. Of note, there was some incongruence between modelling and HCP decisions, and better understanding is needed of how perceptions and data analysis affect one another.

Risk stratification versus impactibility

Risk stratification models may accurately predict which individuals are at risk of future adverse health outcomes,2–5 allowing resources to be allocated. However, allocation is inefficient because not all patients will be amenable to the offered intervention. A stratum cut-off risks not allocating care to people with lower risk who would be amenable and achieve better outcomes that those at higher risk.44 Additionally, since risk is deemed equal for all people within a stratum, resources are also allocated equally (figure 2A) and those made available for patients who refuse or do not respond to treatment cannot be reallocated to patients who will respond. Therefore, opportunities to maximise care for the most amenable people will be missed. Impactibility modelling provides an extra layer of information that can help predict where, to whom, when and how to target preventive resources and allow weighting of investment (time, resources and costs) towards these individuals, which can improve efficiency. As shown in figure 2B, the likelihood of success for a given intervention is not necessarily determined by risk level, and individuals amenable to a specific intervention, due to their ‘impactibility’, can be found throughout the stratified population.

Figure 2

Use of impactibility modelling enhances identification of individuals most likely respond to preventive care and allows weighted resourcing. (A) Population with a given condition at risk of an outcome over a specific period of time, stratified by risk. (B) After impactibility analysis, different options can be targeted to the most amenable people. The numbers and positions of dots per intervention highlight that the likelihood of treatment success can be found throughout the stratified population and is not necessarily determined by risk level.

Types of models considered

The models described in the literature fell into three key themes: ‘health conditions amenable to care’, ‘PTS modelling’ and ‘comparison or combination with clinical judgement’. In the first theme, we found that changes in practice did not reduce hospital admissions and care, and sometimes increased them.10 18–20 It was suggested in one study that although input on organisational change from modelling was well accepted, it was not well integrated.21 As a result, depression as a factor for unscheduled care in patients with long-term conditions remained unaddressed. This finding might suggest that these models are too similar to risk stratification because they focus on diseases but leave underlying factors, such as psychosocial and socioeconomic factors, insufficiently addressed.21 Bardsley et al10 showed that different ACSCs follow different trends, possibly even at the national or international level, which highlights the need to consider how the population for assessment should be selected.6

The PTS models assessed in this review included a wide range of clinical, social and behavioural factors mainly assessed by logistic regression to assess in whom treatment had been most successful (see online supplemental information pp 33‒43). Repeatedly, the results underscored that considering the highest levels of risk and treatment costs did not equate to high impactibility. For example, Dubard et al concluded that variables related to medication adherence and historical use of care unexplained by disease burden were more important predictors of impactibility than diagnosis, specific events, disease profile and overall costs of care.31 PTS modelling generally led to improved accuracy in care planning, estimation of cost savings, engagement and/or care quality. These findings support moving away from delineated risk groups towards continuous risk predictions.44

The comparison or combination with clinical judgement theme indicated that HCPs are routinely able to access real-time ‘soft intelligence’ about their patients that is not available to modellers.43 However, this approach is subjective, involving perceptions at system, HCP, clinical, patient and social levels.16 Gathering such information can be highly resource intensive and how it informs decisions can depend on the quality and openness of the patient‒provider relationship. The same information for two different patients might be affected by HCP sympathy or aversion, how well the patient is known, perceived patient characteristics or abilities (eg, willingness to participate, language skills or cognitive status) and manageable care needs.16 Impactibility models could have a complementary role in decision making and might improve the individualisation of care management, even with a broad range of therapeutic options.33

Optimisation of impactibility modelling

There are many possible reasons for differences in impact, including urban/rural setting, deprivation, literacy, language barriers, mental-health challenges, behavioural or personality traits and practicalities, such as inflexible work or childcare constraints.35 45–48 The challenge for PHM, therefore, is to identify which interventions are most likely to succeed for an individual based on their wider circumstances and how those interventions may be delivered in a way that is most likely to achieve a positive outcome, thereby closing the impactibility gap (figure 3).

Figure 3

Use of impactibility modelling (step 03) to enhance identification of patients amenable to benefit and likelihood of achieving the triple aim. ACSC, ambulatory care sensitive condition.

To optimise impactibility modelling, large amounts of data are needed on people’s health behaviours and socioeconomic, clinical and environmental statuses, as well as broader data where possible, such as genomic data. Many data are held by private companies but are not always accessible to or affordable for health system analysts. Completeness of data may affect modelling and, for example, are known to be less complete for people with higher levels of deprivation.49 The different modelling approaches have various limitations and benefits (table 1),7 16 18–21 23 27–33 35–38 42 50–53 which might further determine the choice. If these issues can be overcome, impactibility models have potential to reduce the clinical burden in making decisions about resource allocation and improve the accuracy and objectiveness of decision making in PHM.

Table 1

Practical benefits and limitations of different approaches to determining impactibility

Approach	Benefits	Limitations
Health conditions amenable to preventive care (gap analysis)	Diagnosis data are readily available.18–21 23 Programmes are relatively simple to model and implement.18–20 23 Widely available data can be used to identify specific, evidence-based and scalable actions to address gaps in care.50 51 May reduce inequalities, as preventable health conditions are more common in deprived communities.7	Does not factor in psychosocial and behavioural variables, such as willingness or ability to engage with care. Suitable data to assess gaps are rarely available in real-world records.6
Propensity to succeed (behavioural response)	Identifies groups where an intervention is/is not likely to provide benefit, thereby is designed to avoid wasting resources where they are of no benefit.27–32 Care planning strategies are optimised at an individual and/or population level, based on previous behavioural responses to a range of potential interventions.33	Models would be enhanced by including educational, behavioural, psychological, social, economic and/or health information42 but data would need to be consistently recorded and accessible. Requires interventional data rather than retrospective patient data.
Comparison or combination with clinical judgement	Based on ad hoc, real-time information about capacity to access and engage with care.52 53 Healthcare professionals may be able to predict future deterioration in ‘low-risk’ patients with relatively good current health status.36	Highly resource intensive Relies on the quality and openness of the healthcare professional and patient relationship, and the ability of the data to capture this.16 35–38 May perpetuate biases or prejudices.7

Potential biases towards groups that are perceived as likely to respond well to treatment, which could exclude some of the most vulnerable groups, has been identified as an important potential limitation of using impactibility as a PHM tool.6 37 53–56 Thus, it should be borne in mind that the purposes of considering impactibility PHM are to improve access and equity of care and avoid unnecessarily wasting resources on providing additional interventions that are costly and will not benefit the recipients. Resources should be directed towards closing gaps in the evidence55 and using the knowledge to develop better-tailored approaches to more people, possibly in medium-risk and low-risk categories (figure 2). This approach, based on the learning healthcare system model, in which best practice is implemented and updated by expanding knowledge of science, informatics, incentives and culture,57 will provide practical case studies that can support efforts to develop and trial alternative ways of delivering care to meet the needs of people in different circumstances.

To achieve the triple aim using predictive models will require those models to have broad insights on which to base predictions. Additionally, no single strategy used in the studies assessed can conclusively point to what information is required, but all go beyond previous healthcare resource utilisation. Some approaches are more easily adopted, as the data required are more readily available or they are less resource intensive to implement.

Study strengths and limitations

This study had several limitations. Interpreting and comparing the data was difficult due to widespread inconsistency in terminology. Even at the most basic level, ‘high-risk individuals’ was conflated with ‘those most likely to benefit’ in some papers26 58 despite evidence indicating that these can be highly separated groups.5 31 42 The quality of the articles included in this review was not graded. However, as this is a growing area of interest and few studies are available, it is a strength of the study that we were as inclusive as possible. Owing to the substantial differences in approaches to categorising model outputs and in outcome measures and the lack of reporting these in some studies, it was not possible to perform a quantitative analysis. Finally, in order to make the findings most applicable to PHM, we excluded studies of specific diseases. Of note, given the descriptive nature of this review, it was not registered and no protocol was published.

Conclusions

Impactibility builds on other key PHM concepts, such as risk stratification,59 by assessing more qualitatively which people might benefit the most from certain health interventions and when proactive treatment might be appropriate (eg, preventive care before an adverse health event or a programme to prevent hospital readmission). It is important, to note that not all people requiring medical care have the potential to benefit from preventive interventions in a PHM sense. Nevertheless, although limited research is available so far, it seems that impactibility models can augment access to and equity of care when coupled with clinical insights and provide an opportunity to personalise preventive care delivery. Using this approach, it should be possible achieve the triple aim—simultaneously improving the individual experience of care, improving the health of populations and reducing the per capita costs of care for populations. PTS models seem to improve accuracy of selection patients amenable to care, but very few prospective or comparative outcome data from real-world settings are available, and this would be judicious to explore further. Potential confounding factors, such as model implementation, the effects of biases and prejudices and accuracy and availability of relevant data, should be included in these studies. Additionally, better understanding of why hospital admissions for ACSCs have not been reduced as much as anticipated would be beneficial. Disease-focussed applications will be the subject of our future research.