Identifying evidence of the effectiveness of photovoice: a systematic review and meta-analysis of the international healthcare literature.

BACKGROUND: Photovoice (PV) was conceptualized in the early 1990s to engage community members in capturing/communicating their lived experience narratives through photography. However, no meta-analyses in health research have assessed whether PV achieves its purported effects.
METHODS: We carried forward any relevant references from a previous review identifying PV studies before 2008 and searched MEDLINE, Embase, PsycINFO and Cochrane Central Register of Controlled Trials from 2008 up until October 2019. We included both published and grey literature, in any population or context. We assessed quality with the Effective Public Health Practice Project's (EPHPP) tool and pooled studies using the standardized mean difference (SMD) and 95% confidence intervals (CIs).
RESULTS: Twenty-eight studies were included, showing significant post-treatment effects only for health knowledge (SMD, 95% CIs = 0.41, 0.09 to 0.73, n = 16) and community functions (SMD, 95% CIs = 0.22, 0.03 to 0.40, n = 4). Strong heterogeneity was indicated for health knowledge, potentially explained by a larger effect in ethnic minority populations. There was insufficient follow-up data for health knowledge, while in follow-up for community functions the post-treatment effect was lost.
CONCLUSIONS: PV's post-treatment effect on health knowledge did not translate into positive health behaviours or physical and mental health outcomes, longer-term community functions, or health service outcomes.

Introduction

The present systematic review and meta-analysis builds upon our previous work within the co-creation, co-production or co-design of health research. More specifically, in meta-analyses we have demonstrated the effects on a range of health-related outcomes when stakeholders from other sectors are invited to work alongside academics to collaboratively generate knowledge. However, this evidence prevented us from performing a more robust comparison of specific approaches. One such co-creation approach is photovoice (PV), which we have used to reveal the lived narratives of mental illness and recovery for ethnic minority service users in the UK. By asking participants to capture photographs and retell lived experiences, PV can provide an inclusive and accessible alternative, welcoming service users and other stakeholders, to sharing knowledge and discussing salient individual, community and policy concerns. PV achieves this through a creative and effective combination of visual and textual information (often participants are asked to develop personal captions to contextualize their photos).,

Since originally conceptualized in the early 1990s as a tool to engage community members in communicating lived experiences and influence policy to help address their concerns, PV has arguably gained traction as an approach that engages service users in authentic and meaningful co-creation of their healthcare. A seminal review systematically explored the literature before 2008 on PV projects and how engagement impacted upon health outcomes. Identifying 37 articles, the review concluded that despite the lack of intervention evaluation findings (only 31% of the articles) PV appeared to assist in understanding needs/assets and foster individual empowerment and community engagement. This evidence is presented as narrative accounts due to PV’s foundation as an anthropological/ethnographic approach., The review is also 10 years old and will not fully capture more recent research endeavours to measure effects of PV as an intervention in various fields such as education, but also in health., Additionally, there are no meta-analyses to demonstrate whether PV actually achieves its purported effects on health-related outcomes.

For the present review, PROSPERO was initially searched to verify that there were no ongoing systematic review and meta-analyses of this kind. We broadly asked the following review questions:

‘What methods applicable to a health setting using PV (inclusive of similar participant-driven photography-based methods such as photo novellas) have been described and measured in the international healthcare literature?’;

‘Do these approaches have any significant post-intervention/exposure and follow-up effects on various health-related outcomes and, if so, what are the effect magnitudes?’;

‘In which settings and for whom do these approaches work better, and whether any intervention components or methodological aspects might influence effects?’.

Methods

We followed PRISMA guidelines (Online Resource 1) and a protocol was pre-registered on PROSPERO: CRD42020166926.

Searches/screening process

Catalani and Minkler’s systematic review of PV in health and public health identified studies published before January 2008. KH checked for any relevant studies they cited, while searching for up-to-date evidence from 2008 until 7th October 2019. The structured search strategy (see Online Resource 2) was implemented in:

MEDLINE;

Embase;

PsycINFO;

Cochrane Central Register of Controlled Trials.

Two reviewers (KH and OE) independently conducted screening in two stages; first of all titles and abstracts, then potentially relevant full texts. Authors were contacted if papers were unavailable. Reference lists in identified review articles were also checked, while for study protocols the subsequent publication of data was traced. Any differences in decisions were resolved by discussion or a third reviewer (KB). Covidence systematic review software was used for data management/screening.

Eligibility criteria

Types of studies

We included primary studies with quantitative data with sufficient postintervention/exposure data/estimations to compare PV treatment condition(s) with at least one comparison/control condition. This included experimental study designs such as randomized control trials (RCTs), quasi-experimental and pre-post evaluations, and all observational studies such as e.g. any relevant cohort, case–control and cross-sectional studies. We included peer-reviewed and grey literature (such as theses and conference papers), limited to English language.

Domain/population

We included any health conditions, and healthy individuals if the aim was to improve health-related outcomes such as general well-being in the population. No restrictions were implemented on age, or geographical settings.

Intervention/exposure

Articles had to include a description of the intervention components with PV methods either wholly or partially evaluated. Stakeholders other than researchers (e.g. service users, community members, the public, carers, healthcare professionals and students) needed to have been involved in the development or conduct of these approaches. For clarity, if they had been involved in its development or conduct as such, this would therefore not exclude endpoint evaluations of PV represented by external parties’ exposure to photo exhibitions or novels if these displays contained the non-academic stakeholders’ photos and narratives. However, photo-elicitation methods in which photos had been taken solely by researchers (often to trigger interview responses), and therefore without any involvement of non-academic stakeholders to develop the photos, were excluded.

Outcomes

We followed a similar classification scheme to our previous meta-analyses of co-creation, including the following main outcome domains:

Self-efficacy of participants (e.g. diabetes self-management, hepatitis B self-efficacy);

Health behaviour (e.g. physical activity, medication adherence);

Mental health outcomes (e.g. meaning-making, depression);

Physical health outcomes (e.g. body mass index, injury experience);

Community or social functions (e.g. social connectedness, community activism).

In addition, we considered:

Health knowledge (as separate from behaviour, e.g. recall of nutrient intake);

Health and safety procedures/measures (e.g. intent to wear/fit-check respirators).

Data extraction

A form was piloted and amended as necessary, where KH extracted information (checked by OE) to determine the feasibility of conducting meta-analyses and subgroup/sensitivity analyses (see below).

Quality assessments

We used the Effective Public Health Practice Project’s (EPHPP) Quality Assessment Tool for Quantitative Studies (https://www.ephpp.ca/quality-assessment-tool-for-quantitative-studies/)—a validated tool assessing a study within six respective domains (selection bias; study design; confounders; blinding; data collection methods; withdrawals and dropouts). These make up a global rating, with a study considered ‘strong’ overall quality if containing no weak ratings, ‘moderate’ if one weak rating, ‘weak’ if two or more weak ratings.

Furthermore, the EPHPP tool has three questions for intervention implementation and we added another question on whether the intervention followed recommended ‘gold standard’ approaches as originally conceptualized in the PV literature, (i.e. to score ‘yes’ on this question the intervention needed to incorporate at least an introduction session to introduce participants to the project and PV, a photo reflection session (e.g. using photo-trigger questions), and participants contributing to the development of captions for photo display).

Two reviewers (OE and KH) independently assessed quality with any discrepancies resolved through discussion, or adjudicated by a third reviewer (KB).

Data synthesis

A narrative synthesis contextualized the data, while KH conducted (and OE checked) random effects meta-analyses of post-treatment and follow-up data. Standardized mean differences (SMD) with Knapp-Hartung adjustments of 95% confidence intervals (CIs) were based on the t distribution yielding wider and more accurate CIs than the Z distribution. Prioritization was given to self-calculation of raw data, if available. Where reported in other formats such as binary, these were converted to the SMD using verified formula as continuous outcomes provide greater power., For post/pre-test effects, transformations for the SMD considered test correlations. In case of overlapping samples, we prioritized those where photography-based activities made up the largest component—but if not stated, the largest samples. When effect sizes for multiple relevant sub-outcomes within a main domain were available for a study, we computed an average effect size. We considered SMDs < 0.40 a small, 0.40–0.70 a moderate and >0.70 a large effect.

Within- and between-group statistical heterogeneity was assessed through Cochran’s Q, its degrees of freedom (df) and P-value (significance level < 0.05), and the I2-statistic for within-group heterogeneity (≥50% threshold to indicate ‘substantial heterogeneity’). As per guidelines, we also calculated prediction intervals (PIs) for main meta-analyses containing at least 10 studies. Subgroup analyses investigated demographic factors (stakeholder group, socioeconomic status, ethnicity (ethnic minority/majority classifications of the countries of included studies), geographical context, age (adults vs. younger people/children) and gender) and intervention components (whether the evaluation related to participation in PV or simply viewing photos via exhibitions/photo novels, the percentage of the intervention group that received PV, whether intervention consistency was measured and intervention contamination was likely, and whether standard PV guidelines, were followed as outlined above). Sensitivity analyses were conducted on methodological aspects including study design and quality, in addition to an item not pre-specified in our protocol (publication type; as many of the references were grey literature).

The presence of small-study effects was assessed through Egger’s test by KH (checked by OE) for main outcome domains containing a minimum of 10 studies as per guidelines.

Stata version 16.1 was used to conduct all analyses.

Results

Narrative overview

Figure 1 shows the PRISMA diagram of searches/screening. 3815 records were assessed on title and abstract, narrowed down to 100 on full-text. Finally 28 studies provided relevant data,,, all from our updated search between 2008 to October 2019. For reasons for exclusion on full-text, see Online Resource 3.

Fig. 1

PRISMA flow diagram of searches and screening.

Online Resource 4 gives an overview of the 28 included studies. There were eighth randomized controlled trials,,,,,,, but the majority pre-post evaluations (n = 14),,,,,,,,, and the remaining non-randomized intervention studies (n = 3),, or presenting cross-sectional data (n = 3).,, Seventeen studies were reported in peer-reviewed academic journals,,,,,,, six postgraduate theses,,, and five conference abstracts.,,,, Online Resource 5 shows that the majority (n = 19) of studies were assessed as ‘weak’,,,,,,,,, only five of ‘moderate’,,,, and four of ‘strong’ methodological quality,,, (for separate assessment of intervention components, see Online Resource 6).

Online Resource 4 further shows that the large majority of studies were conducted in the USA (n = 21).,,,,,,,, Thirteen studies reported on PV with service users,,,,,,,,,,,,, three with healthcarers.,, Ethnic minority groups were represented in 13 studies.,,,,,,,,,, Twenty studies were with adults,,,,,,,,,,, seven with children/younger people,,,,,, and one study included both adults and children/younger people.

Meta-analyses

Forest plots of post-treatment and follow-up data by main outcome measures have been displayed in Figs 2 and 3, respectively (for more detailed statistics, see Online Resource 7).

Fig. 2

Forest plot of the effects (standardized mean difference) of photovoice on main outcome domains post-treatment.

Fig. 3

Forest plot of the effects (standardized mean difference) of photovoice on main outcome domains after 3 months.

Sixteen studies reported measures of health knowledge (Fig. 2), showing a moderate and significant post-treatment average effect (SMD, 95% CIs = 0.41, 0.09 to 0.73). However, substantial heterogeneity could be detected (I2 = 87.49%; Cochran’s Q = 83.35, 15 df, P = < 0.01). The prediction intervals (−0.75 to 1.57) revealed further that in some populations PV might still have negative impacts. Between-group comparisons in Online Resource 7 show a significantly larger positive effect with ethnic minority groups (P = 0.01), but also for interventions with unclear involvement and where less than 60% received PV (P = 0.02), in studies with non-randomized designs (P = 0.01) and of weaker quality (P = 0.04). No small study effects were detected (P = 0.15; funnel plot in Online Resource 8).

For self-efficacy, the post-treatment effect was non-significant (SMD, 95% CIs = 0.15, −0.19 to 0.50, n = 11, Fig. 2), with substantial heterogeneity (I2 = 76.88%; Cochran’s Q = 47.23, 10 df, P = < 0.01) and wide-spanning prediction intervals (−0.84 to 1.15). No significant between-group (Online Resource 7) or small study effects (P = 0.93; funnel plot in Online Resource 8) were found. Additional analyses (Fig. 3) permitted by three studies with 3 months follow-up data (SMD, 95% CIs = 0.11, −0.11 to 0.32) confirmed the overall non-significant post-treatment finding.

Moreover, the effect for health behaviour post-treatment was non-significant (SMD, 95% CIs = 0.17, −0.12 to 0.45, n = 7, Fig. 2), with no ‘substantial heterogeneity’ (I2 = 34.84%; Cochran’s Q = 9.16, 6 df, P = 0.16). Sensitivity analyses indicated a significantly higher mean effect size for weak quality studies (Online Resource 7).

Again a non-significant effect could be observed for mental health post-treatment (SMD, 95% CIs = 0.23, −0.28 to 0.73, n = 4, Fig. 2) and at 3 months’ follow-up (SMD, 95% CIs = 0.22, −0.62 to 1.07, n = 3, Fig. 3). For post-treatment, all statistics suggested very limited influence of heterogeneity (I2 = < 0.01%; Cochran’s Q = 3.01, 3 df, P = 0.39).

Turning to physical health outcomes, another non-significant result was yielded post-treatment (SMD, 95% CIs = 0.03, −0.52 to 0.58, n = 2, Fig. 2), with relative consistency between populations around the mean effect size (I2 = < 0.01%; Cochran’s Q = 0.13, 1 df, P = 0.72). Following previous convention, we did not conduct any subgroup/sensitivity analyses as only two studies contained relevant data.

A small significant positive effect was revealed for community or social functions post-treatment (SMD, 95% CIs = 0.22, 0.03 to 0.40, n = 4, Fig. 2), with relative consistency between populations of this finding (I2 = < 0.01%; Cochran’s Q = 0.64, 3 df, P = 0.89). However, the significant post-treatment effect was lost in 3 months’ follow-up (SMD, 95% CIs = 0.25, −0.12 to 0.61, n = 2, Fig. 3).

Finally, health and safety outcomes generated a non-significant post-treatment finding (SMD, 95% CIs = 0.19, −2.26 to 2.64, n = 2, Fig. 2), with the I2-statistic revealing no ‘substantial heterogeneity’ (I2 = 32.69%; Cochran’s Q = 1.49, 1 df, P = 0.22).

Discussion

Main finding of this study

Our systematic review and meta-analyses covered 28 studies from the international literature. It demonstrated that PV methods may be effective in particular (although substantial heterogeneity was shown) by improving health knowledge. Subgroup analyses indicated that disadvantaged ethnic minority groups might especially be benefactors. Inequalities affecting these groups disproportionately might be expressively uncovered through the accessibility of a non-verbal elicitation process using visual techniques of photography and captions, inviting marginalised communities to consider everyday concerns previously not shared such as low engagement with mainstream services due to mistrust.

However, follow-up data were not available and the post-treatment effects on health knowledge did not appear to translate into self-efficacy, or actual health behaviour or ‘observable’ physical and mental health outcomes. On a broader level, community functions were improved post-treatment but not sustained 3 months after, while health and safety on a systems level also seemed unaffected.

What is already known on this topic

This is to our knowledge, through a comprehensive search capturing previous reviews and checking PROSPERO for ongoing reviews, the first systematic review and meta-analysis of the effects of PV in the international healthcare literature. In this respect, we can only compare it in the subsequent section with systematic reviews that have narratively described findings from individual PV studies.

What this study adds

Insofar as the post-treatment effects on health knowledge are grounds for improving a sense of individual empowerment through understanding and contextualization of one’s circumstances, our analyses provide statistical evidence to substantiate narrative reports of previous reviews. Additionally, community engagement is often highlighted as a positive aspect of PV in the same reviews. We also provide more nuance with subgroup and sensitivity analyses teasing out which settings, for whom and in which ways PV might work, and any available data on longer-term effects.

Limitations of this study

We believe that particularly four potential limitations are worthy of further discussion.

Firstly, we did not want to duplicate previous efforts and relied on Catalani and Minkler’s systematic review for the identification of studies prior to 2008. Interestingly however, none of these were relevant for the present review. In fact, the earliest studies fulfilling our eligibility criteria were not conducted before 2012.,,,,, This might reflect an increased use of PV as an intervention to improve health-related conditions and measure effects (e.g. in the field of education), rather than mostly as a qualitative, preparatory tool for identifying pre-existing needs.

Another caveat is the limited number (and generally low quality) of available studies for certain outcomes (e.g. physical health, health and safety). While significant subgroup/sensitivity analyses could reveal a stronger effect size in one group, these findings are observational rather than revealing causal relationships as such.

Thirdly, interventions with unclear involvement and in which <60% received PV demonstrated larger effects, so PV might not generate effects alone. This should be more carefully distilled in future and, if other intervention ingredients are necessary, PV could be adapted and offered as a first line intervention with more intensive interventions complementing it or following suit. Recent literature shows that meaningful engagement of service users and the public is more likely through sustained involvement. So, perhaps the crucial issue is how PV also relies on a broader co-creation framework to yield experience-near identification of challenges and any tailored solutions. Yet, attention ought to be given to successful PV components. Our analyses indicated no differences of whether studies used ‘gold standard’ PV approaches, and whether evaluations related to actual participation in PV or simply viewing exhibitions/reading photo novels. Further investigations were hindered by the limited number of studies and the muddling of intervention components (e.g. education, therapeutic, reflection, discussion components, etc.) and/or failure to sufficiently tease out their relative effects in many studies.

Finally, for PV’s optimal public health application stakeholders should also consider why positive changes in health knowledge were not necessarily mirrored in behaviours. To potentially explain this, health behaviour models vary in focus but increasingly acknowledge a mix of environmental and personal characteristics. To begin with, we cannot lose sight of the large inequalities not only in terms of health (e.g. disease incidence and severity, access to health services and quality care), but also wider socioeconomic and related inequalities potentially influencing engagement in certain health behaviours. By the same token, despite any enhanced knowledge of negative consequences, such knowledge might consciously or more subconsciously be deemed of secondary importance to various ‘temptations’ to engaging in unhealthy behaviours (e.g. alcohol drinking, fast food options) to help cope with challenging life circumstances. These circumstances can influence individuals’ confidence or belief that they are capable of preventing a particular condition in the first place—or by conceptualising individuals’ sense of self-efficacy (which also returned a non-significant result) as a possible bridging function for actionable behaviours.

Previous approaches leaning more towards the radical traditions of community participation and emancipation for social justice of PV, would seemingly use PV more purposefully to provide foundations and encouragement for action. However, overall this review and meta-analysis has also revealed that more research is needed where PV interventions deliberately target pre-existing barriers (perceived or actual), self-efficacy and health behaviours, with built-in mechanisms to foster more positive changes in one domain resulting from another (e.g. from health knowledge, to self-efficacy, health behaviours, improvements in physical or mental health outcomes at an individual and a wider, community level to reduce health inequalities).

Conclusion

This systematic review and meta-analysis has shown that PV can improve health knowledge, but further research is needed on how PV might also generate positive health behaviour, physical and mental health, longer-term community functions and wider health service outcomes, especially for marginalized communities.

Funding

This work was supported by the Lankelly Chase Foundation, which is funding the emerging work of the Synergi Collaborative Centre (a 5-year national initiative to build a knowledge hub on ethnic inequalities and multiple disadvantage in severe mental illness in the UK). The funder of the study had no role in study design, data collection, data analysis, data interpretation, writing of the report or in the decision to submit the article for publication.

Conflicts of interest

The authors have no relevant financial or non-financial interests to disclose.

Author contributions

KH was involved in the conception and KH, OE and KB in the design of the review. KH designed and conducted the searches. KH and OE screened records for inclusion. KH extracted data from primary studies included for meta-analyses, checked by OE. OE and KH performed quality assessment, with differences to be reconciled by KB. KH conducted the meta-analyses. KH drafted the manuscript. OE, KB, RM and MH contributed to commenting on drafts and suggesting any revisions and all authors approved the final version.

Ethics approval

No ethics approval was required as this is a systematic review and meta-analysis of previous literature.

Data availability

The data underlying this article are available in the article and in its online supplementary material.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.