Value, transparency, and inclusion: A values-based study of patient involvement in musculoskeletal research.

BACKGROUND: Patient and public involvement work (PPI) is essential to good research practice. Existing research indicates that PPI offers benefits to research design, conduct, communication, and implementation of findings. Understanding how PPI works and its value helps to provide information about best practice and highlight areas for further development. This study used a values-based approach to reporting PPI at a Research Unit focused on musculoskeletal conditions within a UK medical school.
METHODS: The study was conducted between October 2019 and January 2020 using Gradinger's value system framework as a theoretical basis. The framework comprises three value systems each containing five clusters. All PPI members and researchers who had attended PPI groups were invited to participate. Participants completed a structured questionnaire based on the value system framework; PPI members also provided further information through telephone interviews. Data were deductively analysed using a framework approach with data mapped onto value systems.
RESULTS: Twelve PPI members and 17 researchers took part. Views about PPI activity mapped onto all three value systems. PPI members felt empowered to provide their views, and that their opinions were valued by researchers. It was important to PPI members that they were able to 'give back' and to do something positive with their experiences. Researchers would have liked the groups to be more representative of the wider population, patients highlighted that groups could include more younger members. Researchers recognised the value of PPI, and the study highlighted areas where researchers members might benefit from further awareness.
CONCLUSIONS: Three areas for development were identified: (i) facilitating researcher engagement in training about the value and importance of PPI in research; (ii) support for researchers to reflect on the role that PPI plays in transparency of healthcare research; (iii) work to further explore and address aspects of diversity and inclusion in PPI.

Introduction

Patient and public involvement (PPI) is considered to be a key part of good research practice [1, 2]. In health research it is increasingly recommended that patient representatives are involved at all stages, including identification of research priorities, study design, study management and oversight, communication of results, and planning next steps [3]. PPI helps to ensure that patients’ voices are reflected and addressed in the design and delivery of health research and helps the outputs to be relevant and meaningful to them [4].

Recognition of the importance and value of PPI in research has grown substantially in the last decade. Funding bodies such as the UK National Institute for Health Research (NIHR) include a mandatory section on PPI activities as part of the grant application process in applied research. The UK Department of Health and Social Care supports PPI through funding INVOLVE [1], a national advisory group that supports active public involvement in the NHS, public health, and social care research; INVOLVE provides guidance on best practice. In 2020 the NIHR has launched a new Centre for Engagement and Dissemination which brings together activities in PPI, engagement, and participation with its strengths in research dissemination [5].

Literature on the impact of PPI demonstrates benefit to enrolment and retention in clinical trials [6], study design [7, 8], and diffusion of health-care innovation [9]. However, some have suggested it is challenging to objectively evidence the full scope of PPI impact because the wide range of activities and levels of patient involvement used in research can make it hard to evidence [10, 11]. In addition, early guidance about evaluation of PPI has focussed on practical elements of the process and lessons for good practice [8, 12], with limited evaluation of theoretical components [13, 14].

Discussions of PPI have often focused on reasons for including PPI in research. Recent literature has highlighted three themes: morality, transparency, and efficiency. First, patients have a right to have a voice in the research work that is conducted either about them or about treatment of their health, and that researchers have a moral obligation to redress any power imbalance in research and healthcare decision making through PPI [15-17]. Second, that PPI takes research outside of academia and into the real world [18, 19], which enables accountability and increases transparency [20]. Third, commentators write that inclusion of patients and members of the public in research planning and conduct makes it more efficient, more valuable, and means that findings are more likely to be successfully implemented in real-world settings [21-23].

In assessing the impact of PPI, it has been suggested that understanding both the theoretical aspects of PPI and the practical application of patient involvement work provides a fuller picture of how involvement functions [24]. Using a theoretical framework to underpin reporting of PPI is one way to achieve this, enabling the scope to review the value of PPI and to identify any areas for further development. A recent systematic review by Greenhalgh and colleagues identified the existence of a ‘plethora’ of frameworks, tools, and guidelines that exist to support, evaluate, or report PPI [25]. Based on the review, the authors developed a taxonomy of five main categories of framework used in evaluation: power-focused, priority-setting, study-focused, report-focused, and partnership-focused. Importantly Greenhalgh and colleagues recommended that researchers consider which framework is the best fit for their purposes, and make decisions accordingly, ideally in collaboration with stakeholders.

PPI and the Musculoskeletal Research Unit

To deliver PPI within the Musculoskeletal Research Unit at the University of Bristol, UK, the ‘Patient Experience Partnership in Research’ (PEP-R) group was established in 2010. PEP-R comprises patients with experience of musculoskeletal conditions and treatment. Since 2010, PEP-R has been expanded to reflect growth in research at the Unit, and at the time of writing three groups run alongside one another to provide specialised input into research studies that relate best to their own experience. Groups currently comprise: a group addressing a variety of musculoskeletal conditions and care (PEP-R), and two specialised groups addressing pain (PEP-R STAR) and infection after joint replacement (PEP-R INFORM).

A focused evaluation of the PEP-R group looking at the impact of patient involvement was conducted between 2011–2012, after the first/original PEP-R group had met ten times [26]. The evaluation was particularly focused on process-related aspects of the PEP-R group. On the basis of that evaluation we worked to improve research projects’ feedback to the group on how their input had helped to shaped studies as well as enabling studies to engage as early as possible with the group. We wanted to use a theoretical framework to develop our understanding of PPI once the PEP-R activities were well embedded into the Research Unit and the two specialised groups had been running for several years, we therefore carried out a study of the three PEP-R groups at the Unit in 2019–2020. Following recommendations from Greenhalgh and colleagues we reviewed the existing evidence-based approaches to reporting PPI. We identified Gradinger’s framework as appropriate and fit for purpose because of the focus on value systems that would enable us to explore meaning as well as process. Gradinger’s work identifies value systems in public involvement and comprises three ‘broad value systems’: i) normative perspectives concerning ethical and political aspects of public involvement, ii) substantive perspectives relating to the consequences of public involvement, and iii) process values concerning conduct within public involvement for example, respect and trust and honesty. Each of these values are further split into five sub-clusters, which can be seen on Table 1. When used to understand involvement with patients and researchers, the framework gives a structured approach to describe and compare the values that occur in processes and activities of PPI in the PEP-R groups.

Table 1

Mapping to Gradinger’s value system.

Value system	Value cluster	Data
Normative	Empowerment	PEP-R questionnaires and interviews
	Rights	PEP-R questionnaires and interviews
		Researcher questionnaires
	Change/Action	PEP-R questionnaires and interviews
		Researcher questionnaires
	Accountability/transparency	No data
	Ethical Values	PEP-R questionnaires and interviews
		Researcher questionnaires
Substantive values	Effectiveness	PEP-R questionnaires and interviews
		Researcher questionnaires
	Quality/relevance	No data
	Validity/reliability	No data
	Representativeness/objectivity/generalisability	PEP-R questionnaires and interviews
		Researcher questionnaires
	Evidence base	PEP-R questionnaires and interviews
		Researcher questionnaires
Process values	Partnership/equality	PEP-R questionnaires and interviews
		Researcher questionnaires
	Respect/trust	PEP-R questionnaires and interviews
	Openness/honesty	No data
	Independence	No data
	Clarity	PEP-R questionnaires and interview
		Researcher questionnaires

Materials and methods

The study comprised questionnaires with researchers and clinical academics, and questionnaires and interviews with members of the PEP-R groups. Data collection took place from September 2019—January 2020. Ethical approval was granted by the University of Bristol Faculty of Health Sciences Research Ethics Committee in 2019 (reference: 91543, approval date: 03/09/2019) and all participants provided informed consent to take part. The design of the study was discussed with PEP-R including identification of their preferred methods for data collection. Group members suggested that additional telephone calls would enable them to expand on their questionnaire responses. Therefore, telephone calls were included in data collection after they had completed questionnaires. Initial results were discussed with three members of the PEP-R group and all three felt that the results accurately reflected their views. Study results were presented at a PEP-R meeting as a means of sense-checking (respondent validation) the work. The group members thought that the results were insightful and of importance. We also sense-checked results with researchers at a Research Unit presentation, they thought that the results were a reflection of their experiences.

Data collection

To design the questionnaire we used Gradinger’s framework [27] with items tailored to the context in which the PEP-R groups were working so that questions made sense to them. Members of PEP-R were also asked if they would like to take part in a short telephone interview after completing the questionnaire, with these interviews designed to give participants the chance to speak in more detail about their views and experiences. Questionnaire responses for PEP-R members were recorded on paper and via email and returned to the research team. The interview topic guide is included as a S1 Table. Interviews were audio-recorded and transcribed verbatim. For researchers, completion was through Online Surveys, a portal that allows completion and collation of results online. Questionnaire responses from both groups and interview transcripts were uploaded to NVivo for analysis.

Participants

The study included two groups of participants: (1) staff members comprising researchers and clinical, research active, academics with experiences of attending PEP-R groups at the Musculoskeletal Research Unit, (referred to as ‘researchers’) and (2) members of the PEP-R groups. Researchers and clinical academics who had attended PEP-R meetings during the previous three years were invited to complete the questionnaire. Responses were anonymous so that participants felt able to provide candid feedback. Overall, 24 researchers were invited and 17 completed the questionnaire. All 17 PEP-R members were invited to complete the questionnaire. Twelve members returned questionnaires, 10 on paper and 2 electronically. All twelve who returned the questionnaire also took part in telephone interviews. PEP-R members were aged between 49 and 79 years old, and all had been members of the group for between 4–9 years. Due to the small sample size a detailed breakdown of demographics is not provided in order to maintain anonymity. We do not provide demographic details of the researchers because we asked them to provide information anonymously, this was because our consultation with researchers during the design of the study indicated that researchers would feel more open and candid to share their views if anonymous, and collection of even straightforward characteristics had the potential to identify them.

Analysis

Questionnaire data and interview transcripts were imported into the qualitative software package NVivo 10. Data were deductively coded against constructs from Gradinger’s Framework. Two transcripts were double coded and the data team worked together to compare data and enhance triangulation. We used framework approach as a means to structure the process in which we assigned code labels to portions of the data and then worked as a team to make sense of the material as a whole [28]. In brief, data were coded based on the three overarching value systems, each containing five clusters as shown on Table 1. Two researchers coded the data (AB & EB). Coding was discussed and refined within the research team to maximise rigour [29].

Results

Data coded onto all three value systems, encompassing 11/15 value clusters. An overview of the mapping is shown in Table 1.

Normative values: Concerning ethical and political aspects of public involvement

Empowerment

Empowerment values relate to transfer of control, self-help, and the right to representation and accountability. Views of PEP-R group members primarily focused on representation and whether they felt empowered to share their views. Group members reported feeling confident to share their views in meetings.

“Basically, it’s an open discussion, where I feel you can give your views, my views, whether it’s ridiculous or not. It’s quite open basically.” (Patient 5)

Some members acknowledged that it took time for their confidence to develop and feel secure in expressing their views.

“I feel more comfortable now to make suggestions, but I was terribly nervous at first!” (Patient 4)

Other members, particularly those who had been impacted by their health, thought that involvement in the group helped them to regain their confidence by engaging with others in similar situations. They described feeling empowered to contribute based on their own experiences.

“It gave me my confidence back again … when you’re at home after you have the operation it’s a horrible feeling of being on your own. You don’t know what’s going to happen next, if anything…. So yes, it’s been nice to see other people, how they managed.” (Patient 4)

Rights

Rights values relate to the intrinsic value of PPI and the right to have a say. In addition to feeling able to express their views, PEP-R members felt that their voices were valued and that their contributions to research projects had an impact.

“researchers come back and talk to us again and … tell us what’s happened after talking to us the previous time, and how they’ve taken on board what we’ve said, and they’ve changed … things. It does seem to me like we are listened to.” (Patient 3)

Researchers also recognised the value of the group, but this was balanced by comments about the requirement to engage in PPI activity due to their job role or the expectations of research funders.

“It was a requirement of my role but also an education and a positive pressure.”

(Researcher 7)

“[It’s a] requirement of the funding body.” (Researcher 15)

Researchers discussed how to listen to patients’ views. Some researchers felt that patients had more of an empowered voice in specific PEP-R groups than they did as patient representatives within study steering groups. However, their own preferences did not always map onto the expectations of funders.

“PEP-R tend to prefer taking research to the patient rather than joining as co-applicants on the [trial management groups], but we get a lot of pressure from [funders] to include as co-applicants.” (Researcher 9)

“I think sometimes they can feel overwhelmed and disempowered by the conversations that occur between academics.” (Researcher 8 about steering groups)

Change/Action

Change/action values relate to generation and translation of knowledge into action to bring about change. PEP-R members showed high awareness of the impact the group had on research and the changes that were made. In particular, PEP-RI members made frequent reference to changes to documentation and plain English summaries.

“The change that we’ve seen is the way some of the publications have gone out. Where before it was quite medically worded, now as we said put it in plain English so patients can understand. I think that’s one of the biggest changes I’ve seen.” (Patient 11)

“Well … there was a [patient information] leaflet. When we first look at them, we think, “No, that’s not.… It doesn’t make much sense,” … there are also things that get added that we feel were important that haven’t been included [in the first instance].” (Patient 10)

PEP-R members were glad to receive feedback from researchers on the changes that had been made based on their input. One member reflected on longer-term dissemination work and how research participants wish to be kept informed, even after projects end.

“It’s quite important though, if you’ve got patients [who have] spent time being part of the research, it is important that those people do get some proper feedback on the outcome of the research. If that’s been promised I think it’s important to make sure it happens.” (Patient 8)

Ethical values

Ethical values relate to awareness of the need to protect PPI members from potential harms. Two aspects were discussed, the role of the group in providing emotional support and minimising potential distress and ensuring that researchers are keeping to good practice when conducting research.

Patient feedback focused on the social support they received from being in the group.

“We’ve all had different experiences, we’ve all gone through different things, but we all understand where we’re all coming from. So I think it is the support of the others in the group.” (Patient 11)

Some PEP-R members also talked about how being involved in the group had helped them process difficult experiences they had been through with their health conditions and create a positive experience.

“I’ve been through it and it is the most horrible thing that you can go through. If something good and something positive comes out of it like one way is better than the other that is just brilliant … it’s more than worthwhile.” (Patient 11)

Researchers focused on methodology and rigour. Researchers reported that involving PEP-R groups helped them to review their methods and make their research and patient-facing documentation better.

“The feedback I gained in the groups directly impacted on both of my projects. I made changes to planned methodology, such as the length of data collection and formats of data collection. I made changes to the plain English summary and patient information booklet.” (Researcher 11)

Substantive value systems: Focused on concerns about the consequences of PPI in research

Effectiveness

Effectiveness values relate to PPI having an effect on research and implementation. Discussion with PEP-R members focused on awareness of the wider implications of their input, beyond specific research projects, and how their input influenced dissemination activities. Group members felt that their own experiences had value and impact in terms of the feedback they could provide to researchers and health professionals.

“Because we are the general public, we know what we went through, what would help us, what would have improved our wellbeing. We can say to the researchers and the surgeons, “If you’d done x, y and z it would have helped.” (Patient 11)

However, the real-world impact of their input and feedback was less clear to them.

“Whether it results, then, in other people kind of changing what they would, like doctors in other hospitals; whether they would change what they do because of it? That’s the bit that was probably the grey area.” (Patient 9)

This was also echoed in discussions about impact on the wider scientific impact.

“Really, I’m not sure about the scientific impact that I would have.” (Patient 3)

Researchers also responded with comment about the effectiveness of PEP-R, through reflection on its impact on their work and the changes that they had made, this particularly centred on the details of the research design and associated data collection methods.

“I made changes to planned methodology, such as the length of data collection and formats of data collection.” (Researcher 10)

Representativeness/Objectivity/Generalisability

This value relates to creating representative, objective, and generalisable knowledge through PPI. Patient group members and researchers were asked whether they felt the group was representative of the general population and whether it was diverse.

Views on the diversity of the group varied substantially between researchers and patient members. Diversity meant different things to each group. Researchers focused on the group demographics and whether they reflected the diversity of the patient population. They also talked about the level of experience that the group had, particularly in terms of benefits and drawbacks. Researchers appreciated the expertise of the group in terms of understanding research processes and what was needed from them at different stages of the research process. However, researchers also highlighted that this level of experience meant that the group had an elevated level of understanding compared to the general patient population, for example in relation to research language and methodology.

“All members are now very expert in research. This is helpful but also not a true reflection of patient understanding in many cases.” (Researcher 11)

One researcher discussed the inclusion of patients in steering group meetings and the impact of having one expert patient.

“The drawback is that there’s a risk the patient is treated as a research expert, or that one person’s experience becomes the benchmark by which important decisions are made. I’m not saying that’s happened, it’s just a risk.” (Researcher 10)

Patients’ views about diversity focused on individual experience. Most considered the group to be diverse due to the range of experience and individual differences and saw value in these.

“We’ve had just about everything going, I think, and ongoing infections, so, yes, we’re covered in just about every spectrum I think.” (Patient 10)

Some patients highlighted that the age range of the group did not include many younger people. They wondered whether the working commitments of younger people might inhibit their ability to participate.

“I’d like to see younger members of the group … that’s always been my hope that we got a few more youngsters, but it’s asking a lot. I mean, unless you’re in a job where they can also see the value of it and will give you the time off, it’s not easy.”

(Patient 6)

Others also recognised the age range of the group but thought that this was appropriate given that the focus of the group was health conditions and procedures that affected older people more.

“We are a fairly diverse group … the sort of research that we’re involved in, hip and knee replacements and all that sort of thing tends to be something that happens, generally, to older people. So, the group is going to tend to be older. […] that’s probably because younger people tend not to have hip and knee replacements unless they’re very unlucky.” (Patient 8)

Process value systems: Focussed on concerns about the conduct of PPI in research

Partnership/Equality

Partnership/equality value relates to the sharing of power of decisions in equal, reciprocal, and collaborative PPI processes. Patient feedback focused on the internal dynamics of the group and the balance between members, issues around shared decision making with the researchers, and views on group responsibility.

Group members reported high levels of equality and feeling supported and respected in sharing their views.

“Especially between the members, you know, I know that anything I say is respected.” (Patient 9)

“(the researchers) are all very friendly, and it’s easy to say what you think. Nobody makes you feel that you’re being foolish because of your lack of knowledge about the subject.” (Patient 8)

Many of the patients had been part of the group for a long time. Group members and researchers thought that this stability was an important consideration with pros and cons. This included the comfort that came from membership of a stable group, that was balanced by the potential challenge of integrating new members into an established group.

“I can see an argument that says you need some sort of turnover of membership in the group. There are pros and cons of course, and I … feel increasingly comfortable in expressing an opinion with the people you know. Whereas, if you don’t, then, perhaps, a bit more hesitant. On the other hand, if you’ve got a stable group like this and you introduce one new member that new member is bound to feel a little bit of an outsider, for a while at any rate.” (Patient 8)

When talking about achievement of consensus, patients talked about how opinions differed and how views were taken into account.

“Not a big conflict, no. There were differences of opinion, but no.” (Patient 9)

“Quite often the researchers will just take what we say, and I think then they change whatever we suggest. It’s rare that anybody disagrees with us or says, ‘No, we can’t do that,’ or whichever.” (Patient 10)

However, some group members did talk about feeling a sense of hierarchy in the group and said that this meant that they felt less able to contribute.

“I do feel I’m a bit less intelligent than some of them that are there. You know, there are teachers and all sorts there, whereas I only went to a secondary school and didn’t get any exams or anything, so sometimes I do feel as though I don’t contribute enough…” (Patient 7)

However, feelings of difference between group members and the researchers was thought to be expected because of different degrees of research methodology and health knowledge (being ‘better at it’) between researchers and group members. For instance, one member spoke about this in relation to researchers and clinicians.

“Well, I suppose we aren’t equal because they’re much better at it. But I don’t feel minor to them if you know what I mean.” (Patient 7)

Feelings of differing levels of responsibility were also evident in decision-making. Group members did not think that it was their role to make decisions but rather to provide researchers with their views and opinions.

“Future patients are in their hands, aren’t they, so it’s their (researchers’) decision.” (Patient 1)

“I think, as a group, we’re involved in the decision-making only because we are putting forward our thoughts. Obviously it’s up to the researchers to use that in any further research, or, perhaps, make them think about, “Oh, I hadn’t thought about that.” As I said, I don’t feel that we are making decisions as a group jointly at all. We’re not taking a vote on anything. We’re just free to express our own personal opinions.” (Patient 3)

Respect/Trust

Respect/trust values relate to respecting diversity, values, skills, knowledge, and experience in mutually beneficial PPI processes. Researchers and group members felt that, without exception, there was respect and trust within the group.

“When we are all in the room we all respect each other, everybody is there to listen to each other, we all have our own opinions and we all respect each other’s opinions and that is where the trust comes in. The trust of it is we know we can say whatever we’re feeling, and nobody will judge.” (Patient 11).

Group members reported feeling that all views and opinions were respected both within the group and by researchers who took part in the group.

“I know that anything I say is respected … my comments are taken on board and, with anybody, with the other people in the group or the researchers. I don’t ever feel any sort of disrespect at all.” (Patient 10)

“They [researchers] do treat us as equals and always show that they value our opinion.” (Patient 7)

Clarity

Clarity values relate to clarity of the purpose, processes, communication and definition of PPI. Group members and researchers demonstrated great clarity in the purpose of PPI. Researchers highlighted the value of involving patients’ perspectives.

“The group tends to have a different perspective to those of the researchers, and what a clinician considers important ends up being different to the patients.”

(Researcher 15)

The group valued clear communication about how PPI input was being used. As within the change/action value, feedback on any changes that had been made was key.

“people come with their new ideas and their research, and I think they really do listen. And they come back again and say, ‘Right, we’ve taken stock of this, and now we’ve done it this way.’ you do feel really quite chuffed I suppose… at the end; you think, ‘Oh right,’ you know, ‘they have changed that,’ yes.” (Patient 6)

Group members talked about the importance of feedback about their input This focused on receiving information about how their views had influenced the research and that this maintained group member’s interest and engagement.

Discussion

This study provides insight into PPI work at a Research Unit mapped against Gradinger’s value system framework [27]. Results show that PPI work mapped onto all three value systems, including eleven out of fifteen clusters.

PEP-R members felt empowered to share their own experiences and views on research projects. The supportive nature of the group and rapport between the members helped them gain confidence over time and feel valued by the researchers. These findings resonate with the morality-based rationale for PPI in research, particularly the right to ‘have a say’ [15-17]. However, this was implicit rather than explicit as although members of PEP-R said that they felt confident to voice their opinions in PPI activities they did not articulate strong feelings about their ‘right’ do so. Instead, they spoke about the importance of feeling valued and appreciated, and that they were doing something positive with their own experiences.

Researchers’ views were mixed and predominantly aligned with the efficiency values of PPI work [21-23]. Researchers recognised the value of PPI, especially in reviewing patient documents and addressing methodological challenges. However, some felt that the main reason for their engagement in PPI activities was because of the requirements of research funders in the grant application process. Similar differences between patient and researcher motivations have been found in other studies. For instance, in a recent evaluation of PPI in a randomised trial of urinary tract treatments patient motivations focused on making a positive difference and supporting future patients, whereas researcher motivations were focused on improving research methods and enhancing their career opportunities [30]. In-depth qualitative work on health researchers’ attitudes towards PPI has further highlighted differences in approach and value given to PPI, ranging from cynicism to ambivalence to positivity [31, 32]. Boyland and colleagues suggest that disciplinary background may be a factor and that hierarchy-based disciplines may place less value on patient perspectives, or even feel that their own expertise was undermined [31]. In addition, their work highlighted that responsibility for PPI responsibility often fell to more junior research staff who may not have the power either to embed PPI within research or to ensure that feedback is acted upon.

Our study highlighted group members and researchers had different views about diversity of the group. PEP-R’s views on diversity centred on the value of individual health experiences and age range of group members, particularly the relative absence of younger people as current group members are between 49 and 79 years old. However, this reflects the typical age for joint replacement in the UK, with National Joint Registry 2020 showing that the median age for patients undergoing hip replacement is 69 years (interquartile range 61–76 years) and for knee replacement is 70 years (interquartile range 63–76 years) [33].

Researchers spoke about sociodemographic diversity and they suggested that they would value inclusion of members with a wider range of socio-economic and cultural backgrounds, and ethnicities. This was framed both in terms of whether the PPI group was representative of the wider population, but also the need to include underserved groups in research projects and PPI work. This is of particular importance in light of concerns that lower socioeconomic status and health inequality is linked with poorer health outcomes [34-36], including musculoskeletal conditions [37, 38]. Our work highlights the difference between a need to include diverse voices in PPI work and inclusion in research. We see these concepts as different but related. Inclusive PPI enables underserved voices to be amplified and heard in the processes of research design to ensure that the research meets everyones’ needs. In research studies, representation ensures that the study includes members of the most appropriate population. This means that underrepresented groups are included and that the research findings are ‘generalisable’ from the sample to the population. Although the concept of generalisability in qualitative research is debated, and some prefer to refer to ‘transferability’ [39], all research seeks to generate knowledge of broader relevance and applicability. PPI on the other hand is a process and activity centred on design of research to meet this end goal, and ensuring diversity in this design is a way to improve research. Prioritisation and community-engagement activities that were precursors of PPI are good examples of approaches that seek to ensure inclusion and diversity. For instance, Citizens’ Juries work by promoting active citizenship through processes of ‘deliberate democracy’ [40] and can bring together diverse citizens to discuss issues of public concern, such as health policy decisions [41, 42]. This approach has been used in priority setting for health research [43] and can enable engagement from members of the public [44]. In many ways, PPI takes a similar approach, seeking to include, empower and amplify in order to bring together members of professional and public communities, ideally for the public good.

Providing information on diversity in PPI groups presents ethical challenges around privacy and confidentially, and raises issues relating to disclosure. As the PEP-R group is small, provision of diversity information to external audiences may not be appropriate for reasons of confidentiality. Similarly, researchers who engaged with the PEP-R groups were not provided with access to personal and confidential information about group members’ protected characteristics or circumstances, unless group members choose to disclose such information directly to them. As such, researchers may be basing their well-intentioned feedback about diversity without full knowledge of the group members’ characteristics or circumstances. We consider it to be crucial that inclusion of underserved groups is fostered and prioritised in all aspects of research. We found it interesting that views of what diversity means and the diversity present within PEP-R varied between researchers and group members and suggest that diversity and inclusion in PPI could be a topic for further research.

The importance of diversity in PPI groups has recently been stressed. A 2020 NIHR report has re-emphasised the need for greater equality and diversity in PPI and outlined changes to their PPI information requirements for funding bids [45].

In our study, work commitments and location of the group meetings were highlighted as barriers to inclusion of a wider range of members. Due to the COVID-19 pandemic all PPI activity at our Research Unit moved to remote delivery of group work using video conferencing and speaking to individuals on the telephone. This offers an opportunity to try new ways of enabling patient involvement and offer greater flexibility in timing, format, and geographical location of group members. However, this mode of delivery has brought new challenges as it is important to support group members to feel confident using the technology and to consider whether they have access. For individuals with limited internet or device access this may create further barriers to involvement. Further evaluation of these methods will be needed.

Previous literature on the value of PPI has highlighted its role in increasing transparency and accountability of health care research and policy decision-making [19]. This is vital in the wider healthcare context, and ensures that healthcare decisions and treatments are underpinned by a clear evidence base [46]. Transparency has become a key focus in relation to some healthcare innovations that have caused public concern [47]. For instance, transparency in relation to the implementation of new technologies in surgery has been highlighted in recent high profile legal cases such as vaginal mesh treatment and metal-on-metal hip replacements [48, 49]. In our study, value clusters on transparency lacked data within all three systems. This is surprising given increasing public awareness of the issues surrounding transparency in healthcare. A possible explanation is that transparency was viewed as beyond the scope of the PEP-R groups, which is reflected in discussion related to shared decision making and wider impact within the substantive value system. Arguably, transparency is within the responsibility of researchers and healthcare decision-makers. Communicating effectively and clearly with members of the public about how and why healthcare policy decisions are made, and ensuring high quality PPI, is central to processes of transparency. However, these clusters were also not evident within researchers’ views. This finding suggests that training and development of PPI activities could focus on the value and delivery of transparency.

Recommendations for future PPI

This study has highlighted areas for further development of PPI, both in terms of training opportunities and the constitution of group membership. Key areas may include:

Facilitating time and providing signposting for researchers to engage in training and continued learning about the value and importance of PPI.

Supporting researchers and members of the group to access information and engage in reflection on the role of PPI in ensuring transparency in research.

Work to address dimensions of diversity, while also acknowledging that diversity characteristics or experiences are not always be visible.

This research was designed in collaboration with patients. Members of the PEP-R group provided input and the study design was improved by the addition of interviews based on their views.

The work used an established theoretical framework which enabled exploration of values in the PPI activity.

Research findings reflect views and experiences of patients who were involved in the groups and researchers who had engaged in PPI activities.

The study evaluated an established PPI group with whom researchers had ongoing engagement. To help members of the group and researchers feel able to be open in their responses, data collection was carried out by a new staff member who was fairly unknown to the PEP-R group, but who was still based within the Research Unit team at the time. There was a chance that the new staff member may not have been considered entirely ‘neutral’ and for this reason we conducted the study with enhanced anonymity in the data collection and reporting processes. However, this meant that it was not possible to analyse data in light of personal characteristics or other factors that might lead to identification,

The topic guide based on Gradinger’s framework was not designed or reviewed by patient members as they were taking part in the study. Some patients said that at times they found the theoretical nature of the questions difficult to answer. To address this the interviews with patient members were carried out to enable the researcher to provide respondents with the chance to discuss their views in more detail.

PEP-R interview topic guide.

(PDF)

Click here for additional data file.

5 Jul 2021

PONE-D-21-16524

Value, transparency, and inclusion: Patient involvement in musculoskeletal research

PLOS ONE

Dear Dr. Whale,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The comments of the reviewers are overall positive. Yet, they advice for some revisions that I kindly ask you to consider.

Please submit your revised manuscript by Aug 19 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sara Rubinelli

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please modify the title to ensure that it is meeting PLOS’ guidelines (https://journals.plos.org/plosone/s/submission-guidelines#loc-title). In particular, the title should be "specific, descriptive, concise, and comprehensible to readers outside the field" and in this case we feel it is not informative and specific about your study's scope and methodology.

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Value, transparency, and inclusion: Patient involvement in musculoskeletal research

This is a nice study exploring the views of researchers and patients who are involved in a 'partnership in research' group. It is original research using appropriate methods guided by a relevant theoretical framework. The standard of writing is generally good.

I think the manuscript could be improved with some relatively minor modifications as follows:

Lines 77-79

“However, it is hard to evidence the full scope of PPI impact because the wide range of activities and levels of patient involvement used in research can make it hard to quantify[10, 11].” – Does evidence of impact have to be quantifiable? Perhaps rephrase? Also, avoid using the same terms twice in a sentence (‘hard’).

Lines 86-87

“... researchers have a moral obligation to redress any power imbalance through PPI [15-17].” What power imbalance? In the development of services? In the research or consultation process? Please clarify.

Line 94

“patient involvement work” ?

Lines 119-122

Missing words in this sentence? “We wanted to use a theoretical framework to develop our understanding of PPI once the PEP-R activities were well embedded into the Research Unit and the two specialised groups had been running for several years, [so we???] we carried out a study of the three PPI groups at the Unit 2019 - 2020.”

Line 168

“clinical academics who had attended PPI meetings during the previous three years were” – do you mean PEP-R meetings? In fact, you refer to both PPI and PEP-R groups from here on. It seems you are using them interchangeably? I found that confusing. It seemed to me that you are evaluating your PEP-R group which is an example of PPI.

It would be good if you could provide your questionnaire and interview guide as additional files. This would help readers make sense of why there were clusters with missing data in your framework. Eg. why did you have nothing on accountability/transparency? As you point out, it is surprising.

Lines 189-195

It’s great that you describe this framework but it made for confusing reading. Possibly because the punctuation wasn’t consistent? I found it hard to identify the main construct and the subconstructs. Perhaps you could number the three main constructs? Table 2 shows them clearly – but that comes later.

I think it might be more descriptively accessible to refer to ‘researchers’ rather than ‘staff’ when you ascribe quotes.

Line 224 -223 and lines 289-299

I think these two sections are dealing with the same issue. Can you find a way to discuss them in only one place?

Page 15 – what do staff think about the role of PPI groups in ensuring effectiveness? I find myself wondering if you asked about this. Another reason to supply your questionnaire.

Lines 345-346 – missing words in this sentence?

Line 419 “different degrees of knowledge” knowledge of what? Clinical issues? Research?

Line 426 – “ members did not think that it as their role to make decisions but rather to provide” AS should be WAS

Line 428 - apostrophe - researcher’s should be researchers’

463-476 – can you blend the two sections here? Maybe use one intro and only one of the quotes as it feels repetitious.

548-549 “... or who do not want to be part of a group” How is this changed by online platforms? Can you please clarify.

It’s great that you engage in a discussion about diversity but I’d like to see a more critical engagement with the concept of representativeness / objectivity / generalisability in PPI, perhaps drawing more broadly on the literature on community consultation. Is generalisability (or objectivity) possible? Certainly not in the statistical sense. So what forms of representation are sought, and with what aims?

Thanks for the opportunity to review this manuscript and good luck with the revisions. I look forward to sharing the final paper with colleagues who are setting up a consumer advisory registry in musculoskeletal research.

Reviewer #2: This is a well written manuscript, presented in an intelligible way. I believe that this work is important for and provides good advice for researchers who will be using PPI groups. My only observation would be regarding the data. If it is anonymised then it would be nice if it was more freely available, perhaps included as supplementary information.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

21 Jul 2021

Dear Reviewers,

Thank you for taking time to review our manuscript and provide constructive feedback. We appreciate the time you have taken and we have provided a table detailing action taken for each point raised. We also include an additional file providing the interview topic guide.

We hope these changes will meet with your approval.

Best wishes,

Katie Whale

Submitted filename: Response to reviewers.docx

Click here for additional data file.

26 Aug 2021

PONE-D-21-16524R1

Value, transparency, and inclusion: A values-based study of patient involvement in musculoskeletal research

PLOS ONE

Dear Dr. Whale,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Reviewers are overall satisfied with the revisions. Yet, there are still some minor aspects to consider before deciding for publication.

Please submit your revised manuscript by Oct 10 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sara Rubinelli

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: The revised manuscript has been improved and all comments have now been addressed. There are still one or two grammatical errors that need addressing, but these are very minor.

Lines 96-98

“In assessing the impact of PPI, it has been suggested that understanding theoretical aspects and practical application of patient involvement work provides a fuller picture of how involvement functions”

I’m not sure what this sentence means, please can you clarify.

Lines 99-100

"...one way to achieve this, enabling scope to review the value of PPI and to identify any areas for further development”

Are there words missing from this sentence as it doesn't really make sense?

Lines 101-102

“...identified the existence of a ‘plethora’ of frameworks, tools, and guidelines [? that] exist to support, evaluate, or report PPI”

Are there words missing from this sentence?

This is an interesting, and important piece of work and I will look forward to seeing it published.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

8 Sep 2021

Dear Reviewers,

Thank you for reviewing our revised manuscript. We have addressed the minor grammatical points you have raised and submit a revised version for approval. Full changes are detailed in the response to reviewers table.

Best wishes,

Katie

Submitted filename: Response to reviewers.docx

Click here for additional data file.

15 Nov 2021

Value, transparency, and inclusion: A values-based study of patient involvement in musculoskeletal research

PONE-D-21-16524R2

Dear Dr. Whale,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sara Rubinelli

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

19 Nov 2021

PONE-D-21-16524R2

Value, transparency, and inclusion: A values-based study of patient involvement in musculoskeletal research

Dear Dr. Whale:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Sara Rubinelli

Academic Editor

PLOS ONE