Clinical Impact Research - how to choose experimental or observational intervention study?

BACKGROUND: Interventions directed to individuals by health and social care systems should increase health and welfare of patients and customers. AIMS: This paper aims to present and define a new concept Clinical Impact Research (CIR) and suggest which study design, either randomized controlled trial (RCT) (experimental) or benchmarking controlled trial (BCT) (observational) is recommendable and to consider the feasibility, validity, and generalizability issues in CIR.
METHODS: The new concept is based on a narrative review of the literature and on author's idea that in intervention studies, there is a need to cover comprehensively all the main impact categories and their respective outcomes. The considerations on how to choose the most appropriate study design (RCT or BCT) were based on previous methodological studies on RCTs and BCTs and on author's previous work on the concepts benchmarking controlled trial and system impact research (SIR).
RESULTS: The CIR covers all studies aiming to assess the impact for health and welfare of any health (and integrated social) care or public health intervention directed to an individual. The impact categories are accessibility, quality, equality, effectiveness, safety, and efficiency. Impact is the main concept, and within each impact category, both generic- and context-specific outcome measures are needed. CIR uses RCTs and BCTs.
CONCLUSIONS: CIR should be given a high priority in medical, health care, and health economic research. Clinicians and leaders at all levels of health care can exploit the evidence from CIR. Key messages The new concept of Clinical Impact Research (CIR) is defined as a research field aiming to assess what are the impacts of healthcare and public health interventions targeted to patients or individuals. The term impact refers to all effects caused by the interventions, with particular emphasis on accessibility, quality, equality, effectiveness, safety, and efficiency. CIR uses two study designs: randomized controlled trials (RCTs) (experimental) and benchmarking controlled trials (BCTs) (observational). Suggestions on how to choose between RCT and BCT as the most suitable study design are presented. Simple way of determining the study question in CIR based on the PICO (patient, intervention, control intervention, outcome) framework is presented. CIR creates the scientific basis for clinical decisions. Clinicians and leaders at all levels of health care and those working for public health can use the evidence from CIR for the benefit of patients and the population.

Background

As foremost objectives for health (and integrated social) care and for public health have been named the following: to increase quality, effectiveness, safety, efficiency, and equality of the services (1,2).

The objectives of this paper are to present and define a new concept of Clinical Impact Research (CIR) and to suggest which study design either randomized controlled trial (RCT) (experimental) or benchmarking controlled trial (BCT) (observational) is recommendable, to consider the feasibility, validity, and generalizability issues in the two study designs of CIR, and to present a simple way to formulate the study question in CIR.

Methods

The proposition of using a wider concept of impact research instead of outcomes research is based on narrative review of the literature and on the author’s idea that in intervention studies, there is a need to cover comprehensively all the main impact categories and their respective outcomes to patients or to individuals living in a population. Starting from the study question, one should consider that besides RCT (3), the observational study design is also capable of delivering evidence of impact of interventions, and for some study questions may be the only available study design (4). A previous study presented the novel concept of benchmarking controlled trial (BCT), which covers all observational effectiveness research (4), and in some circumstances, it is suggested to be considered as an alternative to the RCT. BCTs assess difference in effectiveness between single or a set of intervention(s), between clinical pathways, or between interventions targeting healthcare system factors with an aim to increase effectiveness. An earlier study has presented the concept of System Impact Research (SIR), which aims to assess the impact of health or public health system features on individuals (5).

The considerations on how to choose the most appropriate study design (RCT or BCT) when assessing the clinical impacts within the three categories of CIR were based on previous methodological studies (4,6–9), and on author’s previous work on the concepts benchmarking controlled trial and System Impact Research.

Evidence-based medicine framework was used to outline a simple way of presenting the study question in CIR (10).

Results

Clinical Impact Research (CIR) is defined as a research field aiming to assess what are the impacts on patients or on population of healthcare and public health interventions. The term impact refers to all effects caused by the interventions, with particular emphasis on accessibility, quality, equality (of obtaining services of uniform quality), effectiveness, safety (occurrence of adverse effects), and efficiency (cost-effectiveness). Each impact category includes generic and specific outcomes. The term CIR refers to the conceptual basis and goal of the activity, and BCTs and RCTs are the means to achieve as nonbiased evidence as possible to broaden the evidence base of CIR. The six impact categories of accessibility, quality, equality, effectiveness, safety, and efficiency in CIR are presented in Figure 1.

Figure 1.

The six impact categories in Clinical Impact Research.

Figure 2 describes how to choose the most appropriate study design (RCT or BCT) when assessing the clinical impacts within the three categories of CIR: impact of a single or set of interventions (RCT is the primary design), impact of a clinical pathway (BCT is the primary design), and performance of healthcare providers in relation to each other (BCT is the only feasible design).

Figure 2.

Choosing the most appropriate study design when assessing: (1) impact of a single or set of interventions, (2) impact of a clinical pathway, and (3) performance of healthcare providers (in routine healthcare circumstances) in relation to each other. RCT: randomized controlled trial; BCT: benchmarking controlled trial.

Figure 3 proposes how to shape the study question in CIR using the population, intervention, comparator intervention, outcome, study design (PICOS) framework. Documentation of main baseline characteristics and predictive factors of the population or patients, and particularly in observational settings, the features of the healthcare system serve two main purposes. Firstly, description of the study population and the system features allow inferences of the generalizability of the findings. Secondly, differences at patient and system level need to be controlled to make the study groups comparable in terms of predictive factors other than that under study. In RCTs, successful randomization usually controls for both subject-level and system-level characteristics. In benchmarking controlled trials, both subject- and system-level factors are potential confounders (4).

Figure 3.

Shaping the study question of the Clinical Impact Research (CIR) using the PICOS (population, intervention, control intervention, outcome, study design) framework. The outcomes are related to the impact categories of accessibility, quality, equality (of obtaining effective services of uniform quality), effectiveness, safety (occurrence of adverse effects), and efficiency (cost-effectiveness).

Discussion

The present paper proposes that besides benefits, harms, and costs also three other impacts of interventions should routinely be considered in intervention research. The present paper also suggests how to choose experimental or observational study design when assessing impacts of health care and public health interventions targeted to patients or members of a population. Of the six impacts quality (particularly based on competence of the staff, and on the degree of using evidence based interventions) is a prerequisite for equality, effectiveness, safety, and cost-effectiveness.

This paper presents the concept of CIR, which utilizes both effectiveness research categories, the experimental, that is RCT, and the observational, that is, BCT. The study questions can be categorized into three group, namely, single intervention(s), clinical pathway, and performance comparisons between peers. Suggestions on how to choose the most suitable method (RCT or BCT) for a particular study question are presented. Also, a simple framework for assessing the study question based on the PICO-framework is presented.

Randomized controlled trials (RCTs) are acknowledged as studies providing the least biased information of the effectiveness of interventions; usually of single interventions under optimal (experimental) circumstances (6). Observational effectiveness studies always aim to assess effectiveness under ordinary (nonexperimental) healthcare circumstances. These studies utilize comparisons between peers – healthcare providers treating similar patients – and therefore, these studies are called BCTs (1,2). BCT may be a secondary option, a preferable option, for example, when the study question favors BCT, or even the only option due to feasibility reasons or when there are ethical grounds against conducting a RCT.

Major differences between BCTs and RCTs in the risk of bias of the study are related to the selection of patients. In the former, patients entering the study in each treatment arm may differ at baseline due to selection, while in the latter random allocation to treatment arms leads usually to comparable treatment groups (4). Generally, randomization leads to comparability also on system features, but for example, in case there is uneven recruitment from the participating healthcare units, there may be a need for controlling also system features in the statistical analyses. In BCTs, features related to the healthcare system are always potential confounders, which usually have to be adjusted for in the analyses. Suggestions on how to avoid and control these confounders have been presented in a previous paper (4).

Conclusions

The new concept of Clinical Impact Research (CIR) is intended to provide guidance for planning, conducting, and assessing studies aiming to provide evidence of all impacts of health care and public health interventions targeted to individuals. RCTs and BCTs cover the whole domain of Clinical Impact Research. Recommendations for choosing the best method for a particular research question are presented. A simple framework for assessing the study question is presented based on the PICO framework.

Clinical Impact Research – creating the scientific basis for clinical and many policy decisions – should be given a high priority in clinical, public health, and health economic research and their results should be used for improvement activities. The clinicians and leaders at all levels of health (and integrated social) care can use the evidence from CIR for the benefit of the patients and the population.