Patient and public involvement: how much do we spend and what are the benefits?

BACKGROUND AND OBJECTIVES: Patient and public involvement (PPI) is seen as a way of helping to shape health policy and ensure a patient-focused health-care system. While evidence indicates that PPI can improve health-care decision making, it also consumes monetary and non-monetary resources. Given the financial climate, it is important to start thinking about the costs and benefits of PPI and how to evaluate it in economic terms.
DESIGN: We conducted a literature review to assess the potential benefits and costs of involvement and the challenges in carrying out an economic evaluation of PPI.
RESULTS: The benefits of PPI include effects on the design of new projects or services, on NHS governance, on research design and implementation and on citizenship and equity. Economic evaluation of PPI activities is limited. The lack of an appropriate analytical framework, data recording and understanding of the potential costs and benefits of PPI, especially from participants' perspectives, represent serious constraints on the full evaluation of PPI.
CONCLUSIONS: By recognizing the value of PPI, health-care providers and commissioners can embed it more effectively within their organizations. Better knowledge of costs may prompt organizations to effectively plan, execute, evaluate and target resources. This should increase the likelihood of more meaningful activity, avoid tokenism and enhance organizational efficiency and reputation.

Introduction

The emphasis on patient and public involvement (PPI), sometimes known as ‘patient and public engagement’, in different aspects of health‐care activity in the UK and internationally has increased in the last decade.1, 2, 3, 4, 5 This is seen as key to health‐care reform, helping to shape health policy and ensure a patient‐focused health‐care system.6, 7, 8

While there has been extensive participatory activity in the NHS, there is limited knowledge about its impact.9, 10, 11 The international research evidence base underpinning PPI remains partial and lacks consistency, and the focus on costs is limited.1, 12 While better evidence is needed to improve practice in PPI, this needs to be balanced against the costs of doing nothing – potentially poorly designed and inappropriately utilized health services, with sub‐optimal outcomes for patients.13 Failing to take full account of legislation, policy and practice guidance may also lead to unforeseen costs associated with judicial reviews and public inquiries.

Analytical frameworks to help service managers systematically assess both the costs and benefits of PPI could help. They may support them in planning and documenting their activities, so they can more rigorously capture the value of PPI to their service. The way in which people perceive the benefits and costs of involvement in PPI can differ, and this has a large impact on their willingness to take part.14 A better understanding of the real benefits and costs would reduce barriers to participation and increase incentives for people to be involved.15

This paper proposes a framework for evaluating the costs of PPI from both an organizational and participant perspective.

Evaluating the benefits of PPI: lack of appropriate analytical framework

The language of PPI is frequently contested, making it difficult to establish clear aims and objectives when planning, delivering and evaluating activity.5, 16 Broadly, PPI involves ‘working together to promote and support active PPI in health and health care and to strengthen their influence on health‐care decisions, at both the individual and the collective level'.17 Another view sees it in terms of the ways patients and public draw on their experience and influence the development and delivery of health services.11

Both Coulter and Tritter draw attention to the diversity of approaches, determined partly by the overall aim or goal of involvement. In the NHS, the emphasis is often more on the development of structures and mechanisms for PPI than a focused task or purpose around which activity can be appropriately planned.17 Lessons from involvement in other public services are helpful in understanding that there is no single effective approach but a range of methods that need to be selected and applied based on the resources available and linked to a defined and transparent aim.18

A more rigorous approach to PPI could help us to better understand its value. There is much to be gained from working closely with patients, but this must be aligned to improvement in services and outcomes.17, 19, 20, 21, 22

There is only limited debate about where resources should be invested and whether this investment provides an immediate or sustainable return. Public and patient involvement is difficult to value in economic terms for a number of reasons. The variety of approaches, differences in objectives, and distinctions between individual and collective perspectives on its benefits all present practical difficulties. One problem from a financial and economic perspective is that the process of engagement by patients and the public is seen as equally important as the outcomes. This is because it offers insight into the views and perceptions of others, and more potential for changing organizational culture.5 However, these benefits are much less tangible and easier to understand.

The findings from a systematic review by Crawford et al. (2002) support the notion that involving patients has contributed to changes in the provision of services across a range of different settings. Nevertheless, the same study shows that evidence for the effects on use of services, quality of care, satisfaction or health of patients does not exist.23

Although it is relatively easy to identify the benefits of participation in general terms, there is very little detailed analysis of these, with benefits largely assumed or taken for granted.24 This can lead to problems in the design and conduct of PPI.2 Poorly conducted participation can lead to a lack of trust among participants and a loss of reputation for an organization.25 A lack of clarity around the objectives of PPI can result in confusion or risks alienating the people involved if expectations are not managed, increasing costs or leading to sub‐optimal decisions.26

In developing an analytical framework, traditional economic analysis methods were first examined to assess their applicability to PPI.27, 28, 29, 30 Many disciplines, including political science, social science, community development and international development, provide useful perspectives on the costs and benefits of participation within their own field.31 However, PPI does not typically lend itself to traditional or commonly used methods in economic evaluation, which on their own are inadequate for capturing all the potential benefits and would provide only limited insight for much effort (see Table 1).

Table 1

Economic evaluation methods and their limits in patient and public involvement (PPI) evaluation

It is difficult to express outcomes of PPI in a single quantifiable measure of effectiveness or utility, or to capture the range of outcomes generated by PPI within a single measure when there are numerous and heterogeneous effects involved. It is unlikely that outcomes of PPI can be translated into the type of single monetary, effectiveness or utility measures required by traditional methods of economic evaluation, and attempts to do so would be complex and contentiousa:

Cost‐minimization focuses primarily on costs, comparing the costs of programmes with broadly similar outcomes. It has insufficient focus on outcomes to be of use to PPI and it is not appropriate to value participation activities.27, 28 Cost‐benefits analysis measures both costs and benefits in monetary terms and would require the translation of benefits into purely monetary metrics. Cost‐effectiveness compares programmes costs per unit of outcome produced, where outcomes are measured on a single scale.27, 29 This method allows comparison of competing programmes using a single outcome measure (such as an odds‐ratio for surviving a disease) and provides evidence for which one should be funded. Similarly, cost‐utility analysis condenses all benefits into one generic measure of utility (such as a quality‐adjusted life year, or QALY, for a patient).29 There are many methods expressly designed for capturing non‐market values, including production function, hedonistic pricing, stated preference (contingent valuation and choice modelling), balanced scorecard and social return on investment. However, none of these are appropriate for valuing the benefits of PPI.32, 33, 34, 35, 36

For example, it would be very difficult to capture the monetary value of outcomes such as ‘democracy’, ‘legitimacy’ or ‘social cohesion’.32

To develop a suitable evaluation framework for PPI, we need to start exploring which are the potential benefits, risks and cost of involvement. Several instruments have been developed to measure ‘patient centeredness’, but these are unable to provide enough focus on involvement because they attempt to cover so many dimensions.24

Benefits and risks of public involvement

Although most benefits cannot be ‘translated’ into straightforward monetary measures, it is important to be aware of the benefits that PPI may deliver. A literature review suggests there are different types of benefits arising from PPI,37, 38, 39, 40 that we summarize in the following groups:

The effects on NHS governance and legitimacy;

The effects on research design and implementation;

The effects on efficiency in the design and quality of projects;

The effects on participants and citizenship (the so called ‘feel good’ effect);

The effects on equity issues.

Our focus in this paper is on the effects of PPI on improving health services, and we concentrate here on the last three of these groups.

Effects of PPI on efficiency in the design and quality of projects

By incorporating information about needs, priorities and capabilities of local people, PPI can help ensure more appropriate decisions about resource allocations and help ensure services are adapted to local needs and wishes.41 This can engender a sense of ownership and reduce costs by promoting optimal use of services. People are more likely to be informed if they are involved, increasing their understanding of a decision and the probability of its successful implementation.42 Involvement can also raise the enthusiasm of staff, and leading to a more productive working environment and better quality outcomes.43 Greater public involvement in decisions makes it more likely that a project will be sustained.42 Moreover, PPI has the potential to stimulate the development of new ideas and innovative approaches and solutions.41, 44

Effects of PPI on participants and citizenship: the ‘feel good factor’

People involved in decisions about matters affecting their own lives confirm that they are valuable and valued members of society and can contribute in an active way.45, 46 Being active participants in an area of public life, such as health care, can encourage people to participate in other areas, such as the environment.45 Public involvement helps to raise awareness and increase understanding of public institutions and the way they work, enabling people to better access the services they need, and to understand the boundaries and limitations of different public bodies. It obliges people to think more carefully about their preferences and priorities, and about their values and beliefs, and to temper these in the light of public debate.2, 12 By participating, people contribute to a greater sense of social integration, social cohesion and solidarity.46, 47, 48

Effects of PPI on equity

Public participation has an impact in terms of equity. It can bring diverse and sometimes hostile communities together, incorporating ‘hard to reach’ and ‘disadvantaged’ groups into discussions, building relationships between different communities and social groups, strengthening and creating new networks that enable different interests to work together. Marginalized and excluded groups can be given a voice, reducing the risk that some interests are over‐represented. This helps to create greater equality of access to policy and decision‐making processes. Participation increases equity in achieving a balance across different interests groups and ensuring that all needs are expressed and met. This should secure higher equity not only when services are designed, but also when people access the services.

Risks of PPI

Despite the generally positive view of participation, there is concern that it might be an expensive waste of time and money and might increase the risk of pressure from specific interest groups.49, 50 According to Burton, it is a serious mistake to presume that more participation is necessarily better because2

The involvement of large numbers of people can lead to overenthusiasm, without meeting people's expectations;

Poorly conducted participation can be expensive in time and resources, and lead to poor reliability and a lack of trust and conflicts;25

People may oppose the initiative or behave in a self‐interested manner, increasing the costs to manage the rest of the process or leading to suboptimal decisions;26

Voluntary participation can be seen as a free and infinite resource, and local people might be expected to do for free what experts are generally paid for;51

If programmes rely on short‐term funding and depend entirely on the enthusiasm of individuals, they may not be sustainable in the long‐term.52

There is also the risk that PPI might be used as an excuse for not doing more rigorous evaluation, or simply to obtain consent and legitimacy.

Towards a framework for evaluating costs of PPI

Current shortcomings

When evaluating the impacts of PPI, we cannot forget that there are many opportunity costs linked to it. Many PPI practitioners are sceptical of any attempt at ‘valuing’ participation practice and see economic or monetary analysis of the costs and benefits as reductionist or simplistic. When economic analysis of patient involvement activities is performed, recording of costs and benefits is fragmented.53 Insufficient resources are allocated for monitoring and evaluation at the outset and costs are difficult to measure retrospectively. Where efforts are made, it is often difficult to pinpoint the costs of specific PPI activities. The costs of PPI are typically funded from various budgets or badly recorded, making it difficult to identify the real aggregate cost. The iterative nature of PPI also makes economic analysis difficult, with costs incurred at different points in time. The nature of participatory activity may change in the light of experience but initial allocated budgets used to estimate cost may not reflect the final resource use. Costs may be ‘hidden’ by practitioners wanting to invest more in the process by finding resources from other budgets. Consultants supporting PPI exercises may not be willing to share cost data due to commercial confidentiality. In the absence of reliable historical data and a clear benchmark, managers may be reluctant to expose themselves to performance management, creating a disincentive for data collection. Finally, where costs are recorded, it is most likely to be the cost to the commissioning organization, with costs to participants rarely addressed.

While traditional health economics approaches are inappropriate for PPI, it is important to gather improved data collection on both the costs and benefits of PPI. Although the relationship between costs and outcomes cannot be expressed in a neat equation, it is still important to document and compare them to help make costs more transparent and clarify what the investment in PPI delivers. This will help provide a clearer picture for decision makers, enabling them to better assess the cost‐effectiveness or ‘business case’ for PPI and whether and how a PPI initiative benefits patients. This should also help organizations set appropriate budgets to fund PPI activities.

A framework for recording costs

Whose perspective?

Economic evaluation can be undertaken from a number of different perspectives – the perspective of the organization providing the service or intervention, the patients receiving it, both of these, or wider society. The choice of perspective determines what costs and effects to count and how to measure and value them (for example, from an organizational perspective, it would be necessary to include the costs borne by the organization for setting up and managing participatory events; from a patient's perspective, the travel expenditure and the time spent to attend the same events would need to be included).27, 30 As NHS organizations pay for PPI activities in the UK, recording costs from their perspective is clearly important. But excluding patients’ costs risks underestimating the value of patient involvement. An over‐reliance on the enthusiasm and goodwill of individuals risks long‐term sustainability of PPI initiatives.52

A comprehensive societal perspective should incorporate all costs and benefits regardless of who experiences them. It includes all health‐care costs, social services costs, wider spillover costs both across the economy and costs that are incurred by the patient and family. This is important for recognizing the distributional impacts of participation and better understanding of relationship between participation and equity/social justice.

Proposed framework for more effectively capture costs

Through the literature on the costs of participation, we can identify direct monetary costs, non‐monetary costs and risks that in many cases can become a cost for the project management. The direct monetary costs include all the costs that can be directly attributed to events and activities of public involvement. These costs include paid staff time, staff expenses, external staff/consultants, fees to participants, participants’ expenses, training for staff and participants, administration, venue hire, other event costs, monitoring and evaluation fees.54 Non‐monetary costs include the time contributed by participants, including time from other work or activities.

A proposed framework for comprehensive and consistent data collection is illustrated in Table 2. This illustrates the range of data items that will be used to assess cost from both an organizational and patient perspective. In Table 3, a template for data collection and cost item for PPI activities is proposed.

Table 2

Cost assessment and collection

The assessment of the costs involved in the participatory activity should follow a defined path

Identification of the planned activities: all the activities involved in the participation process should be identified and tracked (set up, design, implementation, meetings, advertising, consultation etc); Identification of resources involved in each activity: staff, patients, materials, other cost items; Relevant information about activities should be recorded in a spreadsheet or in a diary to keep track of what has been done, who was involved, when, for how long; A reliable measure should be identified to assess the cost of each item (e.g. time spent by staff for each activity, travel time for travel expenses); Data collection: quantity and value of each item should be collected through direct measurement (where possible), interviews, time sheet, administrative records, diaries; Quantity of resources used should be listed; Unitary cost for each item should be assessed; Total cost for each item is given by the quantity for the unitary cost; Total cost of the participation is the sum of the cost of each activity.

Table 3

Cost assessment and data collection

Cost Item	Identification	Measure	Data collection	Evaluation	Documentation
Monetary costs
Staff time	Number of staff involved/role	Time spent (hours, days, months); full/part time	Interviews, time sheet	Salary/hour for time spent	HR data on salary
External staff, consultants	Number of staff involved	Fees paid	Administrative records	Fees	Invoices
Staff expenses	Number of staff involved	Travel time, overnight stay	Administrative records	Expenses claim	Expenses claim
Participants expenses	Number of people involved	Travel time, overnight stay	Administrative records	Expenses claim	Expenses claim
Participants fees	Number of people involved	Fees paid	Administrative records	Fees	Invoices
Training courses (staff)	Number of people involved	Time spent (hours, days, months); full/part time	Administrative records	Salary/hour for time spent or fees	HR data on salary or invoices
Venue hiring	Venue and data	Cost of hiring	Administrative records	Cost of hiring	Invoices
Catering	Number of catering	Cost of catering	Administrative records	Cost of catering	Invoices
Telephone calls	Number of calls	Telephone cost	Administrative records	Costs of calls/number of calls	Bills
Copies/printings	Quantity	Printing costs	Administrative records	Printing cost (unitary cost/quantity or total cost)	Invoices
Postage	Quantity	Postage cost	Administrative records		Invoices
Non‐monetary costs
Staff time (not paid‐voluntary)	Number of staff involved and qualification	Time spent over job time (hours, days, months)	Interviews, time sheet, diary kept by staff	Salary/hour for time spent	HR data on salary
Participant time	Number of participant involved	Time spent (hours, days, months) in meetings, conferences, preparation, consultation etc	Interviews, diary kept by participant	Opportunity cost of time spent (salary/hour)

Source: our elaboration.

Conclusions

Patient and public involvement in health‐care decision making is an essential activity if new services are to be designed in a way that ensures they secure widespread legitimacy. While attention has been paid to how to engage stakeholders and manage the PPI process, only limited attention has been paid to the effectiveness of PPI. Effectiveness is not simply a measure of the number of patients involved in a PPI exercise; it must also take into account the cost implications of these activities.

While PPI may indeed result in important benefits, it also consumes resources, both in monetary and non‐monetary terms, making it all the more important to evaluate its effectiveness in economic terms and provide benchmarks against which future activity can be tested. However, the lack of an appropriate analytical framework represents a serious constraint for a complete economic evaluation of PPI.

The framework proposed in this article aims to help make more explicit the variety of costs that are typically ‘hidden’ or ignored, particularly costs to patients. It therefore forms the basis for a comprehensive but straightforward and pragmatic way for health service organizations, patient groups and other stakeholders to bring greater clarity to their costs. Greater clarity on both costs and benefits of different approaches to PPI will enable decision makers to more clearly assess the ‘business case’ for PPI from a more informed perspective.

Competing interest

This article presents independent research commissioned by the National Institute for Health Research (NIHR) under the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) programme for North West London. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. There are no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years and no other relationships or activities that could appear to have influenced the submitted work.

Source of funding

This article presents independent research commissioned by the National Institute for Health Research (NIHR) under the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) programme for North West London. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. There are no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years and no other relationships or activities that could appear to have influenced the submitted work.

Contributions

EP had the idea for the article, performed the literature search, drafted, wrote and revised the article and final approved the version to be published. She is the guarantor. DC drafted, wrote and revised the article and final approved the version to be published. RM wrote and revised the article and final approved the version to be published. JB wrote and revised the article and final approved the version to be published.