Involving patient research partners has a significant impact on outcomes research: a responsive evaluation of the international OMERACT conferences.

OBJECTIVE: To assess the inclusion of patients as international research partners in Outcome Measures in Rheumatology (OMERACT) conferences and how this has influenced the scope and conduct of outcomes research in rheumatology.
DESIGN: A thematic content analysis of OMERACT internal documents, publications and conference proceedings, followed by a responsive evaluation including 32 qualitative semistructured interviews.
SETTING: The international, biannual research conference OMERACT 10 (Malaysia, 2010). PARTICIPANTS: Senior researchers (n=10), junior researchers (n=2), representatives of the pharmaceutical industry and regulators (n=2), conference staff (n=2), new patient delegates (n=8) and experienced patient delegates (n=8).
RESULTS: The role of patients evolved over 10 years from a single patient focus group to full participation in all areas of the meeting and inclusion in research group meetings between conferences. Five main categories of impact emerged: widening the research agenda; including patient relevant outcomes in core sets; enhancing patient reported instruments; changing the culture of OMERACT and consequences outside OMERACT. Patient participants identified previously neglected outcome domains such as fatigue, sleep disturbances and flares which prompted collaborative working on new programmes of research. Specific benefits and challenges for patients and professionals were identified, such as personal fulfilment, widening of research interests, difficulties in establishing equal partnerships and concerns about loss of research rigour.
CONCLUSIONS: Including patients as partners in OMERACT conferences has widened its focus and adjusted the way of working. It has resulted in new developments in the research agenda and the use of more patient-relevant outcomes in clinical trials. These collaborations have influenced perceptions and beliefs among many patients and researchers, and led to wider patient involvement as partners in research.

Since 2002, patients have participated as collaborative partners in the biannual conference on Outcome Measures in Rheumatology (OMERACT).

Although the contribution of patients has been praised and there is a widespread call for scientific publications on the impact of engaging with patients, no systematically obtained evidence has been published to support the idea that the structural involvement of patients in research conferences is beneficial.

Our qualitative study reports the combined results of a thematic document analysis and 32 semistructured interviews with all stakeholders including researchers, patient participants and representatives from the pharmaceutical industry and international regulators.

Long-term engagement with arthritis patients in OMERACT conferences has significantly influenced outcome research in the field of rheumatology.

Patients have successfully contributed to the research agenda of OMERACT by identifying new domains that are important for patients, and provided the patient perspective in the development of core outcome measurement sets and the development of patient-reported outcome measures.

Triangulation of the combined review of the document analysis and interviews, together with the active involvement of all team members representing a variety of perspectives in the phase of data analysis and interpretation, has enhanced the validity of the study.

The unique context of the OMERACT conferences may limit the generalisability of the results, so comparable evaluation studies in other conference formats would be worthwhile.

Incorporating the patient perspective in healthcare research is strongly promoted by policy makers,1–4 funding bodies and international regulators. Many theoretical benefits from patient involvement in research have been reported,5–8 such as improving the relevance of research questions, improving the recruitment of study participants, and increasing the chances for funding and dissemination of results. In addition, there is an increasing recognition of the essential role of patients in outcome research.9 The USA Food and Drug Administration has made patient involvement mandatory in the process of the development of patient-reported outcome (PRO) measures,10 11 and in the context of Core Outcome Measures in Effectiveness Trials (COMET), patient contributions are seen as crucial in defining domains that are relevant to include in core outcome measurement sets for clinical trials.12 Development of core outcome sets might lead to less variety of incomparable and inappropriate outcome measures, more patient-oriented endpoints and less bias by selective reporting of only positive or statistically relevant outcomes.13 Core outcome measurement sets may ease the work of systematic reviewers in synthesising the results of multiple studies.14 15 The question is, however, whether these theoretical benefits of patient involvement in outcome research make any difference in practice.

The international group Outcome Measures in Rheumatology (OMERACT), which defines core outcome measurement sets in rheumatic diseases, first included patient participants at its sixth biannual conference in 2002 and has continued to do so. This provides an opportunity to analyse the consequences and address the important question of whether patient participation has resulted in any demonstrable impact on the nature of its research activity.

Patient involvement in OMERACT has been presented as beneficial and the 2002 conference report concluded that ‘the preliminary success of this forum’ was the basis for ‘continued and possibly expanded patient participation at the next OMERACT meeting’.16 Two conferences later, others perceived the involvement of patients as “indicative of the beginning of a paradigm shift in thinking about RA outcomes over the last 5 years.”17 Since then, OMERACT has formulated three principles recognising the essential role of patients in outcome research.18 First, patients’ input is indispensable when defining relevant outcome measures, identifying domains that are important from the perspective of patients, and assessing the feasibility of measurement tools. Second, structural involvement of patients during the whole research process provides face validity. Third, OMERACT intends “to ground theoretical discussions in the lived experience of arthritis, and in concepts which can be readily communicated to patients to help with therapeutic decision making.”18

However, the validity of these arguments has never been substantiated by robust evidence for the effectiveness of patient participation in OMERACT, and it is not clear whether or how this involvement has influenced methodologies, procedures, attitudes and research outcomes. Therefore, the objective of this study is to describe and evaluate the contributions made by patients since OMERACT started implementing structural patient participation in its conferences. We review the impact of patients on the research agenda and the development of PROs and explore how including patients has influenced the culture and structure of the OMERACT conference.

Method

Patient participation in research is a new phenomenon that is often not reported or reflected on in scientific publications. This lack of written sources in the scientific literature complicates the study of the process and impact of patient participation through a review of the relevant literature. A provisory search using PubMed (March 2010) for the terms ‘patient participation’ OR ‘patient involvement’ OR ‘user involvement’ OR ‘consumer involvement’ AND ‘OMERACT’ did not generate any relevant reference. Therefore, we conducted a content analysis of relevant documents (any written material on the topic of patient participation).

Documents are a stable, rich source of contextual information, providing well-grounded data on events or situations at low costs. A sound document analysis is rule-bound, systematic, following a coding process where raw data are aggregated into units describing the content.19 We included OMERACT conference proceedings as published by The Journal of Rheumatology (1992–2010) and ‘grey literature’ such as correspondence, invitations, session reports, emails and OMERACT policy documents. The review focused on the arguments, reception and evolution of patient involvement in OMERACT conferences and the contributions made by patients.

Subsequently, a responsive evaluation took place during OMERACT 10 (Malaysia, 2010) using qualitative interviews with representatives of stakeholders. Responsive evaluation is grounded in the hermeneutic research tradition and is used by social scientists to interpret meanings that participants attribute to a phenomenon, here the history and impact of a decade of patient participation from the perspective of the conference delegates. It samples all stakeholders and does not seek consensus, but respects the plurality of opinions, values and interests. This ensures that no perspective is omitted as the result of an imbalance of power.20

The first author (MW) has been involved in OMERACT since 2002 as a patient participant. He has a rheumatic condition and has been educated as a responsive researcher. Characteristics for a responsive researcher are a multiple partiality and the intent to enhance mutual understanding among all stakeholders. The last author (JK) has been involved in OMERACT since the first conference (1992) and has been the leader of the patient perspective workshop between 2002 and 2012. Having witnessed the involvement of patients first-hand from the very beginning, MW and JK provided useful information to start the research; yet it also alerted them to critically reflect how this engagement influenced the research, and how to prevent bias. Therefore, two independent experts (TA, MK) were added to the team. They had no relations with the OMERACT conference and its participants, and TA acted as a peer-debriefer discussing dilemmas and challenging methodological decisions with MW.21

The first author held 32 semistructured interviews before, during and after the 10th conference (table 1) and included senior (n=10) and junior researchers (n=2), representatives of the pharmacological industry and regulators (n=2), conference staff (n=2), new patient participants (n=8) and experienced patient participants (n=8). The interviewees were invited and informed by email. The patient participants were aware of the purpose of the study through a one-page announcement in the preconference patient pack and were asked for informed consent. In the Netherlands, no ethical approval is required for non-intrusive interviews only.

Table 1

Characteristics of interviewees

	Professionals	Patient research partners
Sex (M:F)	12:4	7:9		Interview code
Professional background or diagnosis	10 practising rheumatologists	10 rheumatoid arthritis
	3 full-time researchers	2 vasculitis
	3 other professionals	2 ankylosing spondylitis
		1 fibromyalgia
		1 gout
Number of OMERACT conferences attended
1	5	8		PA to PF, PO, PP
2	0	3	}	PG to PN
3	1	1
4	4	4
≥5	6	0
Interview in relation to OMERACT conference
Before	2	1
During	8	16
After	6	5
Geographical spread
	6 countries	7 countries
	2 continents	4 continents
Research background
	10 senior researchers			RA to RG, RJ, RK, RY
1 research fellow				RH
1 postdoctoral researcher				RI
2 pharma representatives				DA, DD
2 staff members				DB, DC

OMERACT, Outcome Measures in Rheumatology.

Twenty-eight interviews were recorded, transcribed by an independent secretariat and subjected to a responder check. Three interviews were summarised in a report, one interview took place without protocol and, on request of the interviewee, without recording (PF). One interview was done through Skype (PP). The average duration of the interviews was over 50 min, most of them taking place in the humid open lobby of the conference resort. Twenty-four interviews were held in English, which was not the native language for six of them. Eight interviews were in Dutch.

The interview protocols were slightly different for professionals, new patients and experienced patients. The topics were not only derived from the document analysis but also from four pilot interviews and the personal knowledge of MW and JK and the expertise of TA. The topics dealt with: the expected role of patient participants, their selection, preparation and support, and the expected or provided contribution to the OMERACT conference. ‘Fatigue’ was added as a potential probe because publications had already shown that this topic deserved special attention with regard to our research questions.22–24 Participants with long-term experience in OMERACT were asked retrospectively to describe their memories of the discussions and decisions taken about patient participation before and after 2002. Their recollections might be characterised as ‘oral history’.

Selection of interviewees: At OMERACT 10, a total of 172 delegates participated, 152 professionals and 20 patients. Nine patients attended the conference for the first time. All interviewees, except for one patient participant from the hosting country, were selected by MW and JK following an emergent purposive sampling approach.21 They used a list of attendees provided by the congress agency, covering four of five criteria found to be important (stakeholder background, gender, geographical spread and number of OMERACT conferences attended). The criterion ‘opinion about patient involvement’ was assessed on the basis of authors’ insight of the participant as being ‘positive’ (eg, contributing to the patient perspective workshop or involving partners in own activities), ‘indifferent’ or ‘sceptical’ (eg, resistant, not collaborating with partners). When it became clear during the process of data collection that certain criteria were not well covered, new participants were approached till maximum variation was realised. For example, two interviewees, who were chosen because of their previously reported criticism of involving patients, showed a considerable change in perception of patient involvement in a positive way. For this reason, two more interviews were arranged with professionals who had expressed critical comments during the last conference. Finally, to ensure the opinions of young investigators, two OMERACT Fellows were approached, one undertaking a PhD in translational research and the other a postdoctoral researcher active in clinical research.

Saturation was defined as a repetition of data; theoretical saturation as achieving sufficiently robust empirical data to support and describe the identified themes and main categories. Saturation was discussed and agreed within the research team. In total, the perceptions and experiences of 16 patient participants and 16 professionals were collected (table 1).

Data analysis

A thematic content analysis focused in particular on the reported contributions attributed to patient participants. Coding of the interviews was done separately by MW and an independent second coder (MK) who had never worked with active patient involvement before. This resulted in 211 detailed codes that were then combined into 27 subcategories. During several meetings, the project team, representing various backgrounds, discussed the codes and subcategories from a variety of perspectives and sought natural groupings or categories within the data. Triangulation was used in two different meanings: first, as a means of verifying findings against another source (interview) or another method (document analysis) and of enhancing the validity of the data. Second, as a means of enriching the data collection and improving the face-validity by synthesising findings from the document analysis with the personal memories and experiences of respondents who looked back in time. By doing so, gaps in the document analysis could be filled in.

The relevance and validity of the analysis and interpretation of the data were increased by the involvement of an external expert in qualitative health research (TA) as well as by inviting one of the patients (SC) who attended OMERACT 10 for the first time to join the research team. As a patient research partner,25 she was involved in the coding, data analysis and data interpretation to guarantee the patient perspective. To protect the anonymity of the participants, all quotes are presented in the ‘she’-form. Quotes of professional researchers are indicated by ‘R’ and those of patient research partners (in short: ‘partners’) by ‘P’.

Results

History of patient involvement at OMERACT

OMERACT started in 1992 as an initiative to overcome the problem of widespread and inconsistent use of many different outcome measures in rheumatoid arthritis (RA) clinical trials. The objective was to improve “the accuracy and responsiveness to change of clinically relevant (to patient and clinician) endpoints.”26 Rheumatologists from many countries met in Maastricht and achieved consensus on a core set of outcomes for RA. The RA core set was endorsed by WHO.27 The initial stand-alone conference was sufficiently successful to be followed by conferences in alternate years, continuing the discussion and consensus building about new core sets for other rheumatic diseases and new measurement instruments.

During the fifth OMERACT conference (2000), participants discussed the concept of a minimum clinically important difference (MCID). Based on methodological arguments, a growing interest in PROs emerged, culminating in a spontaneous proposal at the final session to invite patients to the next conference. All participants voted in favour of this proposal.28 The chair of the conference felt confident about the proposal because it had been discussed previously in the organising committee, although no decisions had been taken. Participants of the MCID module argued that patient perspectives should be explored further29 and took responsibility for identifying 11 patients to join OMERACT 6 and to review the RA core set.

Our document analysis revealed the unconditional positive reception of patient delegates at OMERACT conferences, and partners confirmed that concerns regarding their involvement were misplaced. They felt that their reception was extremely welcoming. “There was a tangible feeling of relief and a belief that patients’ views and opinions would be listened to and incorporated into the deliberations.”30 Also, the organisers were excited and called the patient involvement ‘a tremendous success’.31

Between 2002 and 2012, a total of 57 partners with different rheumatic diseases have participated as full delegates with equal voting rights.32 Their role and contributions have developed over time. At the first conference (2002), they formed a homogeneous group of people with RA with little or no experience in scientific research. The level of involvement in the conference in general was relatively low, support was not organised and the number of sessions patients attended was limited. Contributions centred on participation in the workshop discussions about the severity of fatigue and the definition of low disease activity, although there was a keynote speech at the opening ceremony.30 In contrast, by OMERACT 11 (2012), the partners were a heterogeneous group with different rheumatic conditions and different levels of experience, competences and cultural background. They received a preconference information pack and were actively supported by a preconference dinner, a glossary, training sessions and a buddy system. They carried out a variety of tasks similar to professionals such as giving plenary presentations, cochairing breakout sessions, reporting back from breakout sessions and preparing consensus statements. Several partners became coauthors of peer-reviewed publications.

Patient contributions to OMERACT meetings and outcome research

Interviewees reported a variety of contributions made by partners during the conference where they are an integral part of the deliberative and consensus-building process.18 These examples are presented below and compared with the document analyses when appropriate. Because research in the domain of fatigue has been reported as the most illustrative example, the contributions in this area will be described in more detail. Using the methodology described above, we identified five main categories from the comments made during the interviews with OMERACT participants (table 2): contributions to the research agenda; development of core sets; development of PROs; culture of OMERACT; consequences outside OMERACT. Finally, we will highlight some of the challenges that emerged from the interviews.

Table 2

Main and subcategories from the analysis of patient contributions to OMERACT meetings and outcome research since 2002

Impact of a decade of patient involvement in OMERACT
Research agenda	Outcome core sets	Patient-reported outcomes	Culture of OMERACT	Consequences outside OMERACT
Generating challenging ideas Identification of patient-relevant research topics: Well-being Fatigue Sleep disturbance Flares	Identification of patient relevant domains to include in core sets for clinical trials: Fibromyalgia Psoriatic Arthritis Vasculitis Gout MRI MCID Remission	Acceptable, understandable and feasible outcome measures for Monitoring adverse events Work productivity Flares Psychosocial interventions	▸ Attitudes Communication Perceptions Motivation Relational empowerment Personal benefits	Local initiatives Local and national networks of partners EULAR COMET ISDM
	Reality check

OMERACT, Outcome Measures in Rheumatology.

Contributions to the research agenda

From the very beginning, partners had a significant influence on the research agenda in the field of rheumatology by participating in OMERACT workshops and small group discussions. They identified new outcome domains that are relevant from their perspective.33 The first Patient Perspective Workshop, attended by 11 patient participants and 41 professionals, focused on the development of “valid outcome instruments that incorporate the perspective of the patient and to prepare the evidence and arguments for their inclusion in the (RA) core set.”24 The preconference paper pointed out the methodological and political challenges: How to elicit and incorporate preferences of patients in RCT's?32 The workshop had been specifically arranged to support the partner contributions including a preworkshop and a postworkshop meeting. The workshop identified the subjective experiences of RA not encompassed in the RA core set but having important consequences of the disease: a sense of well-being, fatigue and disturbed sleep.24

After the first conference was attended by partners, it became apparent that perspectives of professionals and patients differed and that more research was needed to articulate patient priorities.34–37 Partners emphasised the need for a holistic approach to people with arthritis.30 The acknowledgement of the discordance of perspectives initiated new studies looking into the preferences, opinions and experiences of people with rheumatic diseases22 38 39 and developing patient-derived core sets.40 This made participants more aware of the emerging patient perspective: ‘the whole realm of things we haven't looked at’ (RA). New topics emerged: remission, pain, flares and foot problems. One interviewee clearly stated that partners “inspired me for new projects to study the variety in new productivity outcome measures” (RK).

Case study of fatigue

Since 2002, when partners identified new topics for research, studies have been initiated with the firm involvement of partners in the field of sleep disturbances, flares and well-being. The most progress has been made in fatigue, and the emergence of fatigue as a relevant outcome measure in RA provides an illustrative case history. When asked for the greatest benefit of including partners in OMERACT conferences, interviewees unanimously confirmed that the topic of fatigue would not have been on the research agenda without partners expressing their concerns about fatigue as an often neglected symptom of their disease and without the listening of receptive professionals. One of the partners attending OMERACT 6 recalled

I can't remember who brought up the subject, but someone mentioned fatigue. And that was the occasion when one of the other delegates said ‘well, everybody gets tired’. One patient shot to her feet and said ‘no, it's not, it's not like anything you've ever experienced; it's not tiredness; it's a complete wipe-out’. (PM)

Early descriptions of fatigue at OMERACT 6 and 7 led to substantial qualitative and quantitative research. The first studies investigated the prevalence and severity of fatigue in RA and how patients describe their fatigue.23 41–43 The next step comprised a systematic review of measurement instruments for fatigue44 that explored the rigour of existing measurement tools and the need to develop patient-derived instruments that are trustworthy, capturing the concepts and language of patients. Furthermore, a standardised visual analogue scale, opportunities for electronic gathering of data and exploring mechanisms of fatigue that could guide researchers in the development of effective interventions, were added to the research agenda. New data presented at OMERACT 8 (2006) showed that fatigue is not a consequence of RA, but an independent variable that adds new information to the existing RA core set.45 46 This new perception resulted in the acceptance of fatigue as an important outcome for clinical trials.47 48 Fatigue was subsequently added to the RA core set as a recommended outcome.47

More powerful instruments for measuring fatigue in RA have since been devised and validated, starting from the perspective of the patients.49 50 Outside OMERACT, researchers initiated similar studies, focusing on the communication between patients and health professionals in the consultation room.51

The thematic document analysis provided additional evidence for the statement that, without patients raising their voice at OMERACT 6, fatigue would not have been high on the research agenda. The issue of fatigue was not new for rheumatologists.52––54 Fatigue was a symptom that was regularly reported during clinical consultations, but not incorporated in guidelines for monitoring and managing. Fatigue in ankylosing spondylitis was identified by physicians and incorporated in a disease status questionnaire.55 During OMERACT 3 (1996), delegates carried out a ranking exercise trying to prioritise psychosocial measures in musculoskeletal diseases. The discussion groups identified outcomes such as pain, depression, anxiety and fatigue as major concerns.56 For fatigue, eight examples of measurement instruments were given.57 However, after this workshop, nothing happened for 6 years, until patients raised the urgency of fatigue as a serious symptom.

Retrospectively, professionals admitted that they had a blind spot for fatigue in RA and that only hearing from partners at OMERACT made them change their perception of fatigue as an important outcome

Because when I was working in oncology before, during university training, of course we saw that the patients were lying in bed all day and we knew they were exhausted, call that fatigue. But patients with RA, we were ignorant. (RC)

Another physician involved in OMERACT from the start

We were first discussing on fatigue and to be honest: I never ever had before heard of fatigue being a problem in rheumatology. So it got into my mind and then I got thinking about it and then, when I was back, I asked patients if they felt fatigue and I got nearly a 100% positive response. So it was like a coming out, you know. I listened to the patients before but bringing it to a specific topic, that was really what I learned at OMERACT. (RA)

Development of core outcome measurement sets and PROs

During the first two conferences including patients, the focus of partners’ contributions was on agenda-setting and identifying relevant outcomes for clinical trials. Then partners became gradually involved on different levels in other OMERACT activities, varying from being consulted (eg, in a Delphi process) to full collaboration (as a partner and as a coauthor). They contributed by identifying domains that are relevant for disease-specific core sets for psoriatic arthritis,58 fibromyalgia,59 60 gout61 and vasculitis.62 Furthermore, they contributed to the development of core outcome measurement sets for methodological or clinical concepts such as MCID and remission.63 Partners have also played a role in the assessment of the feasibility of instruments and core sets, one of the three key components of the OMERACT Filter.64

Partners have been helpful in the development of PRO measurement instruments in the field of work productivity, monitoring adverse events,65 flares66 and psychosocial interventions such as self-management programmes.67 At the 2010 conference, during the plenary session on flares in RA, one of the partners gave a personal testimony about the devastating impact of the unpredictable nature of RA. A professional in the audience was surprised and reported: “It demonstrates that the disease activity fluctuates more than we can see in our data: our instruments are more flat, and by the limited frequency of measuring we filter fluctuations out” (RI).

Regulators require strong evidence for the effectiveness of new medicines by demonstrating accurately that they reduce structural progression as well as patient important outcomes. By doing both of these, developing standards for high quality imaging techniques and exploring new PROs and translating them into valid and feasible measures, OMERACT has been extremely advantageous for the negotiations with regulators about the registration and relatively generous reimbursement of new biological agents

I think, to be really honest, the patient involvement process in OMERACT and the changes in outcome measurements and the use of them in the drug tests has made a real difference for so many patients. (RB)

Culture of OMERACT

In spite of the initial unanimous vote to invite patients, some researchers were concerned about changing the layout of the conference

My original expectation of a limited contribution was based on fear that patients were not able to transcend their personal experience and to generalize … new stakeholders often don't have knowledge about clinometric. (RE)

In retrospect, researchers explained that they deferred to the proposal in order to reflect a core principle of OMERACT of not immediately rejecting new ideas: “To respect and listen rather than just react” (RA).

Looking back, the number of professional participants who were in favour of partners at the conference slowly increased: “I was impressed by the very good working flow” (RC). Participants confirmed that the presence of partners has changed their way of thinking and talking. “They made my blind spot visible” (RK) and another professional reported

Now what we have found is, and I changed my view, [be]cause it wasn't only from OMERACT. As I got to know more and more patients, I realized, this sounds stupid because it's so obvious but it wasn't obvious to me, that a patient isn't their disease. A patient is a person who happens to have a disease. What a big difference. Because if you're a person that happens to have a disease, then for example you might have incredible skills in an area that might be very useful to move a clinical trial forward. So once I came to that realization then patient involvement becomes an absolutely obvious and integral part of moving forward. (RA)

Partners improved communication and brought dynamics to the dialogue because they are motivated and constructive, without a personal agenda. At a conference such as OMERACT, where the discussion about methodology may become extremely technical, partners reminded participants of the common goal of the conference by providing a human face of a person living with the condition day by day. Their presence made participants more explicit about the objectives of sessions and more explanatory about the terms and concepts under discussion. Together with a reduced use of jargon, this ‘forced’ simplification resulted in fewer misunderstandings for everyone.

For some professionals, the presence of partners complicated the communication. Some believed that partners slow down the process because they are not familiar with technical issues. Others felt disinclined to say what they wanted to out of respect for partners or hesitated to criticise them. One researcher felt embarrassed in the presence of partners and put her own expertise aside to keep things simple: “Patients didn't sometimes understand the objective of the research, which hindered us” (RK). One of the partners admitted that “it is a thin line between providing input and causing irritation” (PN).

An analysis of the responses of patients attending OMERACT for the first time showed that new partners experienced a significant learning curve and a variety of personal benefits. Results from this study suggest that in fact all participants learnt from the contact with other stakeholders. During this process, participants gained trust, respect and understanding, reflecting the emergence of relational empowerment:

Patients were a kind of sparring partner when I entered a relatively new area. That was fun and did clarify a lot. (RK)

Relational empowerment in the context of health research can be understood as a process in which traditional doctor-patient relationships transform into equal partnerships enabling mutual learning processes.68 All participants become stronger by sharing knowledge and responsibilities, and educating and helping each other.

The reported benefits were easiest to identify at the beginning when the level of involvement was still low. They became more diffuse when partners were structurally involved as full and equal collaborators. One interviewee mentioned ‘a reality check’ as an important benefit of partners attending the conference. For professionals, it offered the opportunity to check the relevance of the scope of their research: are we doing the right things according to patients and are we using the right tools and methods? It is a belief of professionals that this kind of feedback is important to legitimise their research and, together with the belief of partners that without this research no innovations will take place, it strengthened the mutual empowerment of both.

Consequences outside OMERACT

The lessons learnt at OMERACT were noticed by the outside world. Partners returning home after the conference have continued to introduce patient participation in local and national research projects or established networks of patient research partners.69–72 Some delegates published a working framework for incorporating the patient perspective in outcome research.73 With the input from several OMERACT participants, the European League Against Rheumatism developed recommendations for the inclusion of patient representatives in scientific projects.25 Following these recommendations, a new patient reported quality of life instrument for RA was developed and validated.40 74 Based on the experiences of OMERACT, the organising committee of the sixth International Shared Decision Making conference decided in 2011 to invite patient participants.75 In the same year, OMERACT delegates, partners and professionals participated in the second COMET conference, demonstrating how the OMERACT methodology can be utilised in other disease areas.12

Remaining challenges emerging from the interviews

The role and contribution of patient participants have changed over time and procedures for patient selection and support have been developed in order to identify patient participants who are able to make a difference. There is still a debate going on whether patients should be selected through strict criteria such as education, communication skills, attitude and familiarity with scientific research. Some argue that an expert meeting like OMERACT needs expert patients who have extended knowledge about methodologies of outcome research and are able to provide a kind of aggregated patient input. At OMERACT, this group represents a minority of delegates who are reluctant to allocate the same rights and power to partners as to the professionals. The vast majority believes that many patients are able to contribute to an OMERACT conference and emphasise that a heterogeneous group of partners in age, gender, condition, experience and cultural background are advantageous for the conference. They intend to develop full representative participation in all phases of research by including partners in working group activities between conferences. Finally, some participants point out the potential risks of partners who become too experienced. They appreciate the naive input as a patient, with a minimum of preparation and reflection. They assume that as soon as you start thinking about your contribution, you lose the unique, individual perspective and become a patient-expert who aligns too easily with professionals.

Professionals shared the opinion that partners need training, although they reported different ideas about the content and aims of such training. Experienced partners as well as novice researchers felt that any new participant has to learn the OMERACT objectives, culture and procedures first, before they can become fully productive, mostly at the second or third conference. This accords with the expectations of partners who attended OMERACT for the first time.

Overview of findings

These results show that a decade of patient involvement has been successful and had a significant impact on various aspects of outcome research. Perspectives of patients are different from those of health professionals. A broad consensus exists that partners at OMERACT have played a vital role in identifying domains relevant from the perspective of patients and in developing new PROs such as fatigue, sleep quality, flares and work productivity. We have shown that patient involvement at different levels and in different phases improves the quality of outcome research, especially in the area of fatigue. By combining the evidence-based knowledge of researchers and the experiential knowledge of patients, a synthesis of both kinds of knowledge has been achieved and documented. The benefits are assessment tools that accurately measure what really matters to patients, are formulated in understandable language and are user-friendly. Other benefits go beyond improving clinical outcome research and include improved communication, mutual empowerment, changed attitudes and substantial consequences outside OMERACT.

Discussion

We set out to describe and evaluate the contribution of patients as partners in rheumatology outcome research, reviewing their impact on the research agenda and the culture and process of the OMERACT conference. The document analysis provided the recorded facts while the interviews allowed an exploration of the intentions, attitudes and perceived benefits or harms of patient participation that complements the document analysis. Since validated methodologies for demonstrating the impact of collaboration with patients in the context of research are lacking, a responsive interview methodology seemed to be a good approach.

Both the strengths and limitations of this study relate to the personal experience of the first and last authors as participants in the developing process of patient–partner participation. Having witnessed the OMERACT process, actors and concerns of both the patient community and the research community was advantageous during the development of interview protocols, recruitment and selection of respondents and data analysis. For instance, the knowledge of the opinions of other participants made it possible to achieve maximum variation. Also, the active involvement in the support and training of partners created an adequate awareness of the relevant items to include in the study. The drawbacks of this engagement are the risks of subjectivity, blind spots and overidentification or underidentification with particular stakeholders. These risks have been addressed by applying strict quality measures for scientific rigour in qualitative, evaluation research.

The composition of the research team purposely included two external experts in qualitative research and a patient research partner, who was actively involved in the coding of interview transcripts and distilling relevant categories for impact, which reduced the risk of subjectivity. Bias was avoided by the check-coding procedure in the analysis of the transcripts as at least two researchers independently coded each transcript, after which the whole team discussed the codes until consensus was reached. Saturation was also part of the discussion in the whole team. The inclusion of various stakeholder perspectives prevented one-sidedness. No signals were identified to suggest that interviewees have simply given desirable answers, or just been friendly to the interviewer. Some interviewees have been rather critical, reporting several barriers for structural involvement of patients in research, but have always added constructive suggestions for improvement. Peer debriefing by an independent colleague (TA) further helped to prevent bias.

Other limitations relate to the difficulties of demonstrating the ‘impact’ of patient involvement.9 In OMERACT, there is a strong belief that patient participation works, a belief that is nourished by the worldwide transition towards more patient-oriented healthcare and health research.76–79 The assumption, however, that the long-term involvement of patients as equal partners guarantees sustainable inclusion of the patient perspective in outcome research complicates a thorough evaluation and makes it difficult to distinguish between expected, perceived and actual contributions. Many participants, not only partners but also young researchers and other newcomers, are not able to identify their own contribution and may not see how their input is reflected in the final outcomes. Partners reported almost unanimously of not being able to confirm substantial contributions during their participation, but they believed they did. More experienced participants, mostly professionals, were less reluctant in reporting illustrative examples of patient contributions.

In a dialogue and consensus-based conference such as OMERACT, many (f)actors contribute to the final outcomes. A linear causal relation between patient involvement and impact is therefore hard to establish; the processes of involvement are rather influenced by and influencing many (f)actors in a mutually interactive way. We found that when the level of involvement of partners increased from consultation to collaboration, it became harder to solely attribute individual or group contributions to the final outcomes. Because neither partners nor professionals act as a representative of any group or constituency, it remains difficult to determine the influence of particular groups or individuals. Participation proved to be a dynamic process, especially when tasks were equally performed by patients and researchers, and when the dialogue between both took place not only during the official sessions but also in the corridors of the conference. It should be noted that striving towards equality is a normative ideal,80 and fighting inequalities between patients and professionals is and remains an ongoing concern. ‘Equality’ may be seen in two ways: as the formal position of patients at the conference (as full delegates, they had the same voting rights as professional delegates, received the same preconference materials and had access to all conference sessions like all other participants) and as equality of partnership or collaboration in terms of influence on the decision-making process. Given the power inequalities between patients and professionals, the latter is the greater challenge, but nevertheless some of the documents and interviews suggest that it has been achieved to some extent. Although patients remain only indirectly represented in the executive committee (the highest decision-making body), our data support the conclusion that a small number of experienced patients achieved an equal relationship with researchers in their area of interest. They obtained the competences that enabled them to perform all kinds of tasks at the conference, similar to professionals, and provided input that justified coauthorship of peer-reviewed articles. We did not obtain in-depth information about the question to what extent power inequalities between patients and researchers still persist, but we know from the feedback of all respondents, including some fellows and researchers who attended OMERACT for the first time, that some did not feel they were treated equally. To what extent this experience was caused by their status of being a patient or by the status of a new participant is still unknown.

A last obstacle for demonstrating the influence of patient participation is the invisibility of experiential knowledge, often hidden in anecdotal stories. It has an impact that is rarely claimed by patients and is not perceived by professionals. Personal comments are normally not reported because they are not seen as a valuable and valid source of knowledge,81 and yet clear documentation of meetings is required to ensure that patients’ contributions become visible.82 Professionals focus on synthesising data and may not notice that the dialogue with patients works like a reality check, generates new ideas or changes their beliefs, behaviour or perception. When partners appear to simply agree with the results presented at OMERACT, it might look as if they do not have any contribution to make, but in fact they confirm the value of the work under discussion and provide face validity to the process. It is for this reason that most professionals appreciate the feedback and input from partners, although not all are aware of this reason. Realising the importance of such a reality check is beneficial for the management of realistic expectations: do not expect innovative ideas, brilliant suggestions and new concepts when inviting partners to join research. Their contributions are often more subtle and need the attention of a modest and committed researcher to be noticed.

Despite these limitations, we believe that the results presented in this study are relevant and valid. It is undeniable that there is a growing belief that patient involvement has been successful and brought a unique added value to the conference. Even those who were originally among the most sceptical participants now report that they have changed their perception about the expected contribution of patient research partners. This study is conducted within the context of a scientific research conference in the field of rheumatology, a long-term somatic condition. Our ability to generalise the findings is therefore limited and extrapolation to other research contexts or to other conditions should be carried out with care.