Influence of placebo effect in mental disorders research: A systematic review and meta-analysis.

BACKGROUND: Randomized controlled trials (RCT) in mental disorders research commonly use active control groups including psychotherapeutic shams or inactive medication. This meta-analysis assessed whether placebo conditions (active controls) had an effect compared to no treatment or usual care (passive controls).
METHODS: PubMed, Scopus, PsycINFO, PsycARTICLES, Ovid, the Cochrane Central Register of Controlled Trials and Web of Science were searched from inception to April 2021 and reference lists of relevant articles. Three-arm RCTs, including active and passive control groups, were selected. Where individual standardized mean difference (SMD) was calculable, random effects meta-analyses were performed to estimate an overall effect size with 95% confidence intervals (CI) comparing active vs passive controls. Heterogeneity was assessed using I² statistic and meta-regression. Funnel asymmetry was evaluated using Egger's test (Prospero registration: CRD42021242940).
RESULTS: 24 articles with 25 relevant RCTs were included in the review, of which 11 studies were of high risk of bias. There was an improvement in outcomes favouring the placebo conditions, compared to passive controls, overall (25 studies, SMD 0.24, 95% CI 0.06-0.42, I² = 43%) and in subgroups with anxiety (SMD 0.45, 95% CI 0.07-0.84, I² = 59%) or depression (SMD 0.22, 95% CI 0.04-0.39, I² = 0%). Meta-regression did not show a significant explanation for heterogeneity. Egger's test showed no asymmetry (p = .200).
CONCLUSIONS: A small placebo effect was observed in mental disorders research overall, and in patients with anxiety or depression. These findings should be interpreted with caution in the light of heterogeneity and risk of bias.

BACKGROUND

Clinical trials evaluating effects of treatments for mental disorders compare outcomes observed in intervention group with those observed in control group. Control groups in such trials sometimes deploy usual care or no treatment (passive controls), and at other times, they use placebo conditions including sham psychotherapy or inactive medication (active controls). , The placebo conditions have been associated with placebo effect. , , , , Variation in the effect of treatment observed in a clinical trial may be linked to the type of control group used, whether passive or active. , There is a debate about the influence of placebo effect on estimation of treatment effect in mental health clinical trials , , . The magnitude of the placebo effect may vary depending on psychological amenability of participants, the mental disorder and the type of placebo under investigation. It is therefore important to explore the influence of disorder and placebo type in order to decipher the extent of the placebo effect in mental disorders. This systematic review and meta‐analysis determined the magnitude of the placebo effect in active control groups compared to passive control groups, and explored whether this effect differed between disorders and type of placebo condition.

METHODS

This work was pre‐registered in PROSPERO (CRD42021242940, https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021242940 ).

We followed the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) reporting guideline (see Table S1). Reporting of the study conforms to broad EQUATOR guidelines.

Search strategy and selection process

We searched Ovid Medline (1950–2021), Embase (1974–2021), PsycINFO (1806–2021), PsycARTICLES (1806–2021), the Web of Science‐Science Citation Index (1899–2021), Cochrane CENTRAL (1948–2021) in April 2021 and reference lists of known relevant articles. The complete search equation that was used can be found in the Prospero registration protocol (Table S2). In addition to the database searches, hand searches of placebo review papers and the reference lists of the included trials were carried out. There were no language restrictions. The selection process was carried out using the Mendeley citation manager.

Three armed randomized controlled trials (RCTs) with adult participants (18 years or older) diagnosed with, or being primarily treated for the symptoms of a mental disorder, as classified by DSM‐V were included. The placebo needed to be described by the authors as a placebo, sham, nonspecific or equivalent. The no‐treatment condition needed to be usual/routine care, waiting list or no treatment, which was of equivalent duration to the placebo condition. Allocation to intervention, placebo conditions or to no treatment conditions had to be by randomization. Trials were excluded if the disorder was linked to a developmental aetiology (e.g. dementia) or an identified organic aetiology (e.g. substance‐induced persisting amnesia or acquired brain injury), or if the symptoms reported were linked to a singular non‐persistent event (e.g. preoperative anxiety). Substance disorders and sleep disorders were excluded, as were studies that recruited healthy controls for no treatment comparison. Titles and abstracts were reviewed independently by R.F.‐L. and B.R.‐G. for inclusion and potentially relevant full‐text articles were screened by R.F.‐L. and B.R.‐G. independently. When a disagreement occurred, the decision for inclusion was made by consensus.

Data collection process

Articles that met the inclusion criteria were subjected to independent data extraction by two members of the research team (R.F.‐L. and B.R.‐G.) using a data extraction sheet that was collaboratively developed. The primary outcome measure extracted for results (e.g. pre‐ and post‐treatment means and standard deviations) was linked to clinical assessment of the symptoms for the disorder afflicting the participants. When more than one outcome measure was reported, we selected one for meta‐analysis using the following rules in order: standardized measures were prioritized over composite measures; if other‐reported and self‐reported questionnaire‐based measures were both available as outcomes, data from the former were extracted because of the greater objectivity in these measures; when previous rules were met, the measure of preference was the one described by the authors as their main one or the one mainly directed to the psychiatric condition being treated.; when more than one measure met the previous rules, depression measures were prioritized over other measures, because they are the more common and could provide higher power for meta‐analysis. For studies that use dichotomous outcomes, odds ratio were calculated and transformed to standardized mean differences (SMD) using the method developed by Chinn. Data were also extracted on sample characteristics (e.g. participating numbers, demographic composition, diagnostic and clinical characteristics), and study design (e.g. randomization procedure, type and length of intervention, placebo/sham group types and no‐treatment/passive group type). Where possible, we contacted authors for data that were not extractable from the published reports. This proved to be restrictive as this often related to papers published over 30 years prior to our data extraction, making authors difficult to trace or data difficult to obtain. No requests for additional data were met.

Risk of bias assessment

The risk of bias within the trials was assessed separately by two reviewers (R.F.‐L. and B.R.‐G.) with the focus on the effect of assignment to intervention using the second version of the Cochrane risk of bias tool for randomized trials (RoB 2). RoB 2 was suitable to capture the quality of both trials of psychotherapies and drugs. It covers five domains of bias: selection bias, performance bias, detection bias, attrition bias and reporting bias. Within each domain, a series of questions (‘signalling questions’) aim to elicit information about features of the trial that are relevant to risk of bias. A judgement about the risk of bias arising from each domain is proposed by an algorithm, based on answers to the signalling questions. Judgements can be ‘Low’ or ‘High’ risk of bias, or can express ‘Some concerns’. The outcome selected for the RoB assessment was the principal end‐point of the outcomes selected for meta‐analysis.

Data synthesis

We computed the effect in individual studies using SMD for symptoms (continuous outcome) using the Hedge's g estimator with confidence intervals (CI), where suitable data were available. Standard errors were also calculated for continuous outcomes using an appropriate equating method. , All preparatory conversion calculations were made using the escalc function of the metafor package for R. Effect size (ES) signs were inverted when an improvement was reflected in an outcome measure by scoring less. A favourable improvement in symptoms under placebo compared to no‐treatment conditions was indicated with an ES value greater than zero.

For pooling results across studies, we used random effects model combining ESs in the meta R package. We determined the importance of size of placebo effect using Cohen's guidelines for interpretation, with standardized ES of 0.2 considered small, 0.5 considered medium and 0.8 considered large. Heterogeneity was expected across trials, with different disorders and placebos being meta‐analysed. The extent of heterogeneity was estimated using the I² statistic with 25%, 50% and 75% regarded as low, medium and high heterogeneity levels. Subgroup analyses were conducted for psychiatric disorders (anxiety, depression and schizophrenia), type of placebo condition (sham psychotherapy and inactive medication) and methodological quality score (risk of bias). Multivariable meta‐regression was carried out on four moderators (disorder, type of placebo, risk of bias and publication year) to explore their influence on variation in ES. We inspected for funnel asymmetry and Egger's test was applied.

RESULTS

From 40,237 titles and abstracts screened, 136 full‐text articles were considered potentially relevant (Figure 1). Of these, 24 met the eligibility criteria, which included three with binary outcomes , , that were not combinable in meta‐analysis with continuous data and 110 were excluded. The reasons for exclusion were not having a placebo group, not having a passive group studies analysing a singular non‐persistent outcome and studies with a healthy group. Characteristics and quality of the 24 articles (1244 participants: only control groups) included in systematic review, of which 21 were included for meta‐analysis are shown in Table 1. , , , , , , , , , , , , , , , , , , , , , , , The trials were published between 1963 and 2021. Most trials concerned the treatment of adults with either an anxiety disorder , , , or a depressive disorder, , , , , , , , , with the remaining four aimed at treating adults with schizophrenia or a related disorder.

FIGURE 1

Identification of trials for the systematic review of placebo effects in mental health research

TABLE 1

Characteristics of trials included in the systematic review of placebo effects

Authors and year of publication	Country	Disorder	Active control condition (n)	Passive control condition (n)	Treatment length (weeks)	Outcome measure (primary or related to symptoms if no primary stated)	Risk of bias score
Studies included in meta‐analysis
Lick et al., 1975	USA	Snake or Spider Phobia	Irrelevant subliminal image (9)	Wait list (9)	4	Behavioural Approach Test	Some concerns
Rosen et al., 1976	USA	Snake Phobia	Factual information (10)	Wait list (7)	8	Behavioural Approach Test	High
Kirsch et al., 1983	USA	Snake Phobia	Expectancy modification (13)	Wait list (5)	5	Behavioural Approach Test	Some concerns
Klosko et al., 1990	USA	Panic Disorder	Placebo drug (15)	Wait list (15)	15	Anxiety Disorder Interview Schedule	High
Rice et al., 1993	USA	Anxiety Disorder	Pseudo mediation (49)	Wait list (49)	4	State‐Trait Anxiety Inventory	High
Syzmanski & O’Donohue, 1995	USA	Spider Phobia	Factual information (5)	No treatment (7)	2	Behavioural Avoidance Test	Some concerns
White, 1995	UK	Anxiety Disorder	Advice (20)	Wait list (21)	13	Hospital Anxiety and Depression Scale	High
Goldstein et al. 2000	USA	Panic Disorder and Agoraphobia	Credible attention‐placebo (13)	Waiting list (14)	14	Panic Disorder Severity Scale	High
Powers et al., 2004	USA	Claustrophobia	Device to induce relaxation (12)	Wait list (15)	1	Peak Fear Rating to Behavioural Task	Low
Shalev et al., 2012	Israel	PTSD	Placebo drug (18)	Wait list (79)	12	Number with PTSD	Low
Rezaei Ardani A et al, 2017	Iran	PTSD	Placebo +TAU (12)	TAU (12)	8	PTSD Checklist Military Version	High
Klerman et al., 1974	USA	Depression	Placebo drug and HC (18)/Placebo drug and LC (15)	No drug and HC (18)/No drug and LC (16)	34	Number of Relapses	Some concerns
Nandi et al., 1976	India	Depression	Placebo drug (10)	No Treatment (8)	4	Hamilton Rating Scale for Depression	High
Rabkin et al., 1990	USA	Depression	Placebo drug (27)	No Treatment (23)	6	Number of Relapses	Some concerns
Propst et al., 1992	USA	Depression	Non‐directive counselling (10)	Wait list (11)	13	Beck Depression Inventory	High
Serfaty et al., 2009	UK	Depression	Talking control with TAU (28)	TAU (55)	17	Beck Depression Inventory—II	High
Watkins et al., 2009	UK	Depression	Bogus training (19)	Wait list (20)	1	Beck Depression Inventory—II	Low
Moldovan et al., 2012	Romania	Depression	Advice about how to be more organized and Telephone calls (24)	Wait list (24)	4	Beck Depression Inventory	Some concerns
Guest et al., 2018	Australia	Depression	Healthy lifestyle (30)	Publicly accessible claim information (30)	10	Depression, anxiety and stress scales	Some concerns
Pots WT et al., 2017	Netherlands	Depression	Expressive writing (87)	Wait list (87)	12	Center for Epidemiological Studies Depression Scale	Some concerns
Whittaker et al., 1963	UK	Schizophrenia	Placebo (13)	No treatment (13)	10	Number of Relapses	High
Lewis et al., 2002	UK	Schizophrenia	Counselling and TAU (71)	TAU (60)	5	Positive and Negative Syndrome Scale	Low
Rass et al., 2012	USA	Schizophrenia	Watching films and Television with TAU (17)	TAU (11)	10	Composite Measure of Global Cognition	High
Gomarr et al., 2015	Spain	Schizophrenia	Computerized typing program (44)	TAU (43)	2.5	Behavioural Assessment of the Dysexecutive Syndrome	Some concerns

Abbreviations: HC, high contact; LC, low contact; PTSD, post‐traumatic stress disorder; TAU, treatment as usual.

The placebo and the no treatment group allocations were incorporated in the same randomization procedure as the one used for allocation to the active treatment group. The exception was one study for which there was no active treatment. Within the included trials, 575 participants were assigned to the placebos, and 620 participants to the no treatment conditions. Sample sizes for placebo and no treatment conditions were relatively small (mean = 25.14, standard deviation = 21.60). Of the placebos used, eight trials used inactive medications, delivered either as a tablet or liquid. The remaining 17 were psychotherapeutic shams, consisting of either the nonspecific aspects of an active treatment or a plausible attention control. For the no‐treatment conditions, the majority were either on a waiting list (12), received nothing (5) or were in receipt of the same routine care as the placebo group (7).

Risk of bias in individual studies

Overall, 11 (45.8%) included studies were assessed to be at high risk of bias. Nine (37.5%) studies presented some concerns, and four (16.7%) studies were at low risk of bias (Figure 2). The most common methodological flaw that included RCTs presented was the absence of reporting of randomization procedures or the presence of sub‐optimal randomization procedures. Additionally, some studies where at high risk of bias due to a lack of statistical analysis plan or had multiple eligible outcome measures, which put them at high risk of reporting bias. Finally, some studies were at high risk of bias in measurement of the outcome mainly because experimental and active control groups tend to be assessed more often than their passive group counterpart.

FIGURE 2

Risk of bias assessment of individual studies included in the systematic review of placebo effects in mental health research. HC, high interpersonal contact; LC, low interpersonal contact

Overall, active control groups showed statistically significant improvements when compared with the passive groups (SMD = 0.24, 95% CI 0.06–0.42, p = .007, I 2 = 43%) (Figure 3).

FIGURE 3

Forest plot for the overall and disorder‐based subgroup meta‐analysis of studies comparing placebo conditions (active controls) with passive control groups. Standardized Mean Differences (SMD) above 0 represent an improvement of active control groups against passive groups, while values below 0 represent the opposite

Subgroup analyses yielded improvements of active control groups against passive controls for psychotherapeutic sham placebo conditions (SMD = 0.29, 95% CI 0.06–0.46, p = .013, I 2 = 54.3%). Studies that presented some concerns (SMD = 0.33, 95% CI 0.03–0.63, p = .034, I 2 = 50.6%) showed an improvement of active control groups against passive controls (Table 2).

TABLE 2

Placebo type and study quality‐based subgroup meta‐analyses of studies comparing placebo conditions (active controls) with passive control groups

	n	SMD	95% CI	p‐value	I 2 (%)
Type of placebo
Psychotherapeutic sham	17	.29	[0.06; 0.52]	.013	54.3
Inactive medication	8	.14	[−0.12; 0.40]	.278	0
Risk of bias
High	11	.25	[−0.02; 0.52]	.070	28.6
Some concerns	10	.33	[0.03; 0.63]	.034	50.6
Low	4	.04	[−0.44; 0.53]	.865	68.4

Psychotherapeutic sham: placebo, sham or equivalent treatment; inactive medication: inert substance that does not contain an active drug ingredient.

Abbreviations: CI, confidence interval; SMD, standardized mean differences.

Multivariable and univariable meta‐regression analyses were all nonsignificant for most of the predictors included (Table 3).

TABLE 3

Results of univariable and multivariable meta‐regression analysis to explore heterogeneity when comparing active control groups versus passive control groups

Heterogeneity variable	Univariable analysis		Multivariable analysis
	Coefficient (SE)	p value	Coefficient (SE)	p value
Disorder	−0.24 (0.25)	.072	−0.25 (0.13)	.080
Type of placebo	−0.14 (0.22)	.541	−0.29 (0.23)	.228
Risk of bias	0.07 (0.13)	.582	0.00 (0.13)	.983
Publication year	−0.01 (0.01)	.4202	−0.01 (0.01)	.338

Abbreviation: SE, standard error.

Visual inspection of the funnel plot did not reveal a clear presence of asymmetry (see Figure S3). Egger's test of the intercept was performed and yielded non‐significant results (Intercept = 0.598, 95% CI −0.72 to 1.92, t = 0.889, p = .383), suggesting absence of publication and related biases.

DISCUSSION

In this meta‐analysis and systematic review, we found that there is a small overall placebo effect, that is, active control groups showing improvement over passive groups, observed in RCTs of mental health research. Previous research in depression have found that different control groups produce different ESs when compared with active treatment. , , , Although our results are in line with them, such studies did not compute ESs between active and passive control groups, and were limited to depression treatment trials.

Strengths and limitations

The present work was prospectively registered in Prospero. Our results suggest absence of publication bias, both by inspecting the funnel plot and calculating Egger's test. Additionally, the overall results are obtained from a broad search with a large number of studies included, which makes the results more robust. However, we found that 44% of included studies where at high risk of bias and. Univariable and multivariate meta‐regression failed to account for the moderate heterogeneity that we found in the meta‐analysis. Nonetheless, it is important to note that results of multivariate meta‐regression should be taken with caution, because the number of included studies for meta‐analysis is relatively low for this model and can lead to overfitting problems.

A limitation of the present study remains in the definition of the placebo group. Sometimes active interventions could be labelled as placebos by other researches. We made our best effort to systematically categorize in a transparent way which active control groups can be considered placebos or actual interventions, but it is important to take into account that opinions may differ over this.

Subgroup analyses were low powered in some cases (schizophrenia and inactive medication), so results in such subgroups should be taken with caution. Additionally, it would have been of great interest to explore differences between different sub‐disorders of anxiety and depression, as well as different passive and active control groups, but it was not possible because a larger number of studies would have been required to have enough power.

Interpretation and comparison with other studies

Subgroup analyses revealed that trials of anxiety disorders were the more sensitive to placebo effect, demonstrating a moderate effect, with trials of depression showing only a small effect, and no effect in schizophrenia trials. This result is in line with previous evidence on the effect of placebo pills on mental health research, which found that anxiety and depression disorders are the ones that respond stronger to placebo, while schizophrenia shows the lowest response.

Psychotherapeutic shams showed a small placebo effect in subgroup analysis, while we found no placebo significant effect on inactive medication. The placebo effect related to psychotherapeutic sham has been a matter of discussion, since it presents some additional difficulties when implementing it when compared to inactive medication, and evidence about its ES is mixed, although recent meta‐analyses point that different control groups produce different ESs when compared to the active treatment. , Our finding on inactive medication seems not to be in line with evidence regarding the effect of placebo medication. , , However, this can be explained by the low power of our subgroup analysis in inactive medication (k = 8). The low power on inactive medication is due to the fact that there is a lack of trials using passive groups together with the active control group in medical placebo research, which makes it difficult to generalize the result.

CONCLUSIONS

An overall small placebo effect was observed in mental disorders research, both overall and for those receiving sham psychotherapy. This effect was observed in patients with anxiety or depression, but not in schizophrenia trials. These findings should be interpreted with caution in the light of heterogeneity and risk of bias.

CONFLICT OF INTEREST

All authors declare no competing interests.

AUTHOR CONTRIBUTIONS

Rodrigo Fernández‐López and Blanca Riquelme‐Gallego were responsible for data collection and analysis, data extraction and management, and quality assessment. Khalid S. Khan resolved any disagreement between Rodrigo Fernández‐López and Blanca Riquelme‐Gallego Rodrigo Fernández‐López, Aurora Bueno‐Cavanillas and Khalid S. Khan participated in study design, analysis, data interpretation and thoroughly revised and edited the manuscript. All authors have read and agreed to the published version of the manuscript.

Supplementary Material

Click here for additional data file.

Table S2

Click here for additional data file.

Figure S3

Click here for additional data file.