Exploring academics' views on designs, methods, characteristics and outcomes of inclusive health research with people with intellectual disabilities: a modified Delphi study.

BACKGROUND: The British Medical Journal's (BMJ's) patient revolution strives for collaboration with patients in healthcare and health research. This paper studies collaboration with people with intellectual disabilities (ID) in health research, also known as inclusive health research. Currently, transparency and agreement among academics is lacking regarding its main aspects, preventing upscaling of the patient revolution.
OBJECTIVE: This study aims to gain agreement among academics on 3 aspects of inclusive health research for people with ID: (1) designs and methods, (2) most important characteristics and (3) outcomes.
DESIGN: A Delphi study was conducted with academics with experience in inclusive (health) research and on people with ID. The study consisted of 2 sequential questionnaire rounds (n=24; n=17), followed by in-depth interviews (n=10).
RESULTS: Academics agreed on (1) a collaborative approach to be most suitable to inclusive health research, (2) characteristics regarding the accessibility and facilitation of inclusive health research, and (3) several outcomes of inclusive health research for people with ID and healthcare. Other characteristics agreed on included: atmosphere, relationship, engagement, partnership and power. It was stressed that these characteristics ensure meaningful inclusion. Interviewed academics voiced the need for a tool supporting the facilitation and evaluation of inclusive health research. There was ambiguity as to what this tool should comprise and the extent to which it was possible to capture the complex process of inclusive health research. DISCUSSION AND
CONCLUSIONS: This study underlines the need for transparency, facilitation and evaluation of inclusive health research. The need for in-depth interviews after 2 Delphi rounds underlines its complexity and context dependence. To increase process transparency, future research should focus on gaining insight into inclusive health research in its context. A tool could be developed to facilitate and evaluate inclusive health research. This tool will be partially applicable to participatory research in general and thereby upscale the patient revolution. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

The Delphi study questionnaires were seen as too restrictive by academics.

Responding to these concerns, additional in-depth interviews were conducted which were successful in gaining in-depth information on the complexities surrounding inclusive health research and in providing more insight into topics to be explored in future research.

Effort was made to collaborate with co-researchers with intellectual disabilities (ID) in this study; however, this proved to be challenging.

This Delphi study makes a comparison between inclusive health research and participatory research in general and addresses future research purposes for both types of research.

To the best of our knowledge, this Delphi study is the first to approach scientifically from the perspective of academics patients' involvement in health research, and it contributes to the next step in the patient revolution.

Introduction

The British Medical Journal (BMJ) has been promoting the patient revolution since 2013. This revolution encourages collaboration between patients and healthcare professionals and aims to improve the quality of healthcare.1 The next step in this revolution is patients' involvement in health research. This addresses the mismatch between research agendas and patients' needs as well as increasing the sustainability1 of health systems and accountability in healthcare and research.2 3 The patient revolution and participation in health research is of great importance to a vulnerable and marginalised group such as people with intellectual disabilities (ID),4 as it is expected to improve the much needed healthcare they receive. Increasingly more papers are published on the participation of different patient groups in research, such as people with rheumatism,5 acquired brain injury6 and mental health issues,7 as well as on participatory research in general.8 It remains uncertain whether these studies fit the needs of people with ID when they collaborate in health research.

People with ID are identified as experiencing limitations in intellectual functioning and adaptive behaviour originating before the 18th year of life.9 As a result, they face a multitude of health inequities compared with the general population.4 These health inequities and unmet needs regarding their limitations complicate their active participation in health research, also known as inclusive health research.10 11 Inclusive health research is defined as ‘research which includes or involves people with learning disabilities as more than just subjects of [health] research’.10 On the basis of expected outcomes of the patient revolution, inclusive health research could contribute to resolving the health inequities and unmet needs of people with ID.

Despite increasing demands, it remains unknown why some inclusive health research activities succeed while others fail.12–15 The evaluation of inclusive health research is challenged by ethical dilemmas such as inclusion being a right in itself regardless of its contribution to the research process.16 In order to gain a scientific perspective and facilitate inclusive health research,17 knowledge and experiences should be shared among academics on three topics. First, a recent literature review has shown that it is unclear which research designs and methods best fit inclusive health research.11 Second, there is little agreement on the important characteristics of inclusive health research (eg, payment and training of people with ID).12–15 Third, expected or experienced outcomes of inclusive health research, such as improved quality and validity of research or increased empowerment of people with ID, are often not explicitly evaluated or assessed.7 11 18 The views of people with ID are of great importance, and their perspectives will be explored in another study. This study focuses specifically on academics with experience in inclusive (health) research, as many ambiguities exist among them and they have a responsibility in making research inclusive. Therefore, a Delphi study was conducted which aimed to gain academics' agreement on: (1) designs and methods, (2) most important characteristics and (3) outcomes of inclusive health research.

Methods and participants

Delphi method

The Delphi method is commonly used to allow respondents in a certain field of work to achieve agreement on a set of criteria. In several rounds, respondents are independently questioned about their opinion, usually by means of a standardised questionnaire. After each round, the researcher provides the participating respondents with so-called controlled opinion feedback in the form of a summary of the results from the previous round. On the basis of this feedback, the respondents may choose to revise or retain their opinion in the next round. To prevent group pressure or undue influence of dominant individuals, the respondents remain anonymous throughout the course of the study and are not expected to communicate with one another. Respondents are not allowed to participate in the subsequent questionnaire if they did not complete the previous one.19–22 The current Delphi study consisted of two rounds. Figure 1 provides an overview of the study procedure.

Figure 1

Overview of study procedure. ID, intellectual disabilities.

For each round, a standardised questionnaire was designed, programmed as a web survey using the online software survey tool LimeSurvey23 and then language checked by a professional translation service. The first questionnaire ran from 1 August to 8 September 2014, with telephonic follow-ups to non-response after two weeks. The second questionnaire ran from 25 September to 4 October 2014, with telephonic follow-ups to non-response after 1 week. After the two questionnaire rounds, academics' rationales indicated that in-depth interviews were needed.

Patient involvement

This Delphi study is part of a PhD project on inclusive (health) research in which two people with ID collaborated as co-researchers. The characteristics in the questionnaires have been discussed with one of these co-researchers in order to gain her perspective on the matter. Additionally, the results of the study were discussed with both co-researchers. However, an adaptation of the Delphi study methodology accessible to people with ID is currently non-existent, resulting in information not suitable to a Delphi study paper. Moreover, presenting the discussions with both co-researchers goes beyond the scope of this study, as it is aimed at gaining agreement among academics. Reflections by the co-researchers on inclusive research, including this Delphi study, are a work in progress and will be published at a later stage of the PhD project.

Panel

An international panel of academic researchers was assembled. Academics were considered as respondent if they (1) had experience with inclusive (health) research or participatory research and (2) had experience in working with people with ID. Names of potential respondents were acquired from the conference proceedings of the fourth International Association for the Scientific Study of Intellectual and Developmental Disabilities (IASSIDD) European conference in June 2014 (http://www.iassidd.org) and the researchers' network. Additionally, key authors in the field of inclusive research were approached. A total of 42 academics were invited to round I. Details of the researchers are provided in the Results section.

Questionnaire design

The questionnaire for round I was designed on the basis of (1) a previously conducted structured literature review on inclusive health research11 and (2) a literature search on characteristics of inclusive health research. The questionnaire was piloted by means of cognitive interviews with academics (n=3). In a cognitive interview, interviewees are asked to fill in the questionnaire while thinking out loud. The interviewer probes the thoughts of the interviewee with the aim of identifying errors in the questionnaires.24 The interviewees who participated in the cognitive interviews did not participate in the actual Delphi study. The questionnaire for round II included controlled opinion feedback from round I, consisting of bar charts, medians and IQRs25 as well as summaries of academics' rationales. Academics were asked to answer the same questions as in the previous round in the light of this feedback. Both questionnaires contained three main topics:

Designs and methods: academics were asked to assign a list of research designs and methods to either or both of the following two approaches to participatory research:26 (1) collaboration, where researchers and the public work in partnership and make joint decisions and (2) control, where the public has complete decision-making power. This subdivision should lead to insight into whether different research designs and methods lead to different types of inclusive health research. Additionally, academics were allowed to state that they ‘do not know’. Academics were offered the opportunity to provide rationales for their decisions.

Characteristics: academics were asked to rate the importance of a list of characteristics (table 1) of inclusive health research on a five-point Likert scale ranging from very unimportant to very important. With regard to inclusive health research in general, various characteristics within the following five themes were presented: recruitment, accessibility, facilitation, reflection and evaluation. Additionally, several characteristics specific to the approaches collaboration and control were presented, as these were expected to differ because of differences in power dimensions. Academics were asked to provide rationales for their ratings.

Table 1

Summary of characteristics of inclusive health research in the questionnaire

In general
Recruitment
Description of recruitment process Representative sample of people with ID
Accessibility
Accessible information Feedback on accessible information Accessible research outcomes Time provided to discuss information Accessible resources Description of provision of accessible information Description of development of accessible information Task definitions
Facilitation
Acknowledgement of extra resources by funding bodies Sufficient amount of time provided Participation as early as possible Structured and described decision-making process Inclusive distribution of research outcomes Involvement in research agenda setting Attention on helper-relations Inclusive implementation of research outcomes
Reflection
Ethical reflection Reflection on justification
Evaluation
Insight into added value Evaluation on personal learnings Evaluation on mutual learnings
Specific to collaboration approach	Specific to control approach
Open, respectful and confidential atmosphere Relationship with research group Training during participation Partnership and shared decision-making power Power to influence research Training before participation Salary Remuneration Number of people with ID higher than professionals	Open, respectful and confidential atmosphere Engagement with research group Training during participation Power to influence research Training before participation Salary Complete decision-making power Remuneration Number of people with ID higher than professionals

ID, intellectual disabilities.

Outcomes: in round I, academics were asked to name the two most important expected outcomes of inclusive health research for four different stakeholder groups: (1) people with ID, (2) research(ers), (3) healthcare and (4) society. In round II, a list of these outcomes was presented, and academics were asked to rate the importance of each outcome for each of the four stakeholder groups on a five-point Likert scale, again ranging from very unimportant to very important.

In-depth interviews

In round II, academics indicated that the standardised questionnaires limited them in providing the desired depth of information. In order to provide academics the opportunity to elaborate, they were invited to take part in in-depth interviews. To gain a variety of responses, a selection was made from academics who either completed (n=5) or did not complete (n=4) both questionnaires. One academic who participated in the cognitive interviews was also included. Additionally, academics from varying fields of expertise were invited. On the basis of preliminary results from the questionnaires, the three main topics were discussed as well as the expressed need for the development of a tool or guideline for inclusive health research. Interviews took 45 min on average and were conducted via telephone or Skype between 17 October and 5 November 2014.

Data analysis

It was decided that agreement regarding designs or methods of inclusive health research (topic I) was achieved when ≥70% of the academics assigned it to a particular inclusive approach. Two steps were taken in order to decide whether agreement was achieved regarding the most important characteristics and outcomes of inclusive health research (topics II and III). First, the characteristics or outcome had to have a median of 5 (ie, response category very important), indicating a high level of importance of the characteristic or outcome. Second, the characteristic or outcome had to have an IQR of 0, indicating that the majority of the academics agreed on the high level of importance of the characteristic or outcome. All quantitative analyses were conducted in SPSS Statistics 20. For topic II, the only characteristics which reached agreement are presented in tables 4 and 5 for the purpose of readability.

Table 4

Characteristics of inclusive research in general

Theme	Agreement on characteristicsRound I	Agreement on characteristicsRound II
Recruitment	None	Description of recruitment process
Accessibility	Accessible information Accessible research outcomes Time provided to discuss information Accessible resources	Idem round I and: Feedback on accessible information Description of provision of accessible information
Facilitation	Acknowledgement of extra resources by funding bodies Sufficient amount of time provided Participation as early as possible Structured and described decision-making process	Idem round I and: Inclusive distribution of research outcomes
Reflection	None	None
Evaluation	None	None

For each characteristic, agreement was achieved when: median=5 (category very important) and IQR=0.

Table 5

Characteristics of collaboration and control

Approach	Agreement on characteristicsRound I	Agreement on characteristicsRound II
Collaboration	Open, respectful and confidential atmosphere Relationship with research group Partnership and shared decision-making power	Idem round I and: Training and preparation during participation
Control	Open, respectful and confidential atmosphere Engagement with research group Power to influence research	Idem round I and: Training and preparation during participation

For each characteristic, agreement was achieved when: median=5 (category very important) and IQR=0.

Rationales and in-depth interviews were used to provide academics the opportunity to identify and elaborate on important themes within the questionnaire. Themes were identified as important when mentioned more than once. Academics' rationales were analysed for similarities and differences, grouped into themes and summarised. In-depth interviews were transcribed verbatim and imported into ATLAS.ti 7. Transcripts were independently coded by two researchers (TKF and JN) by means of a code book based on a previous literature review11 and preliminary results from the two questionnaires. Interpretations were compared and contrasted. For example, in some cases different codes were used for similar pieces of texts. After discussion, the most appropriate codes were chosen.

Results

The response rate to the first questionnaire was 57% (n=24) and to the second questionnaire 71% (n=17), resulting in a dropout rate of 29%. Reasons for non-response were: time constraints, sick leave and maternity leave. Academics originated from the UK (n=6), the Netherlands (n=6), Ireland (n=3), Canada (n=3), Australia (n=2), South Africa (n=2), the USA (n=1) and Belgium (n=1). Other descriptives can be found in table 2.

Table 2

Descriptives

Field of expertise	Participant number	Number of academics in Round I	Number of academics in Round II	Number of in-depth interviews
Health and disability research Intellectual disability research (n=8) Disability and health (n=2) Epidemiology (n=1) Speech-language pathology (n=1) Health science (n=1) Disability studies (n=4) Community care (n=1)	1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 18, 19, 20, 21, 24	18	12	4
Participatory research Participatory action research (n=1) Inclusive research (n=1) Patient participation (n=1)	8, 16, 23	3	2	2
Mental health research Mental health (n=1) Psychology (n=2)	15, 17, 22	3	3	3
Cognitive interview (n=2)	NA	NA	NA	1
Total		24	17	10

NA, not applicable.

Topic I: designs and methods for inclusive health research

Academics' rationales indicated that advisory boards, self-advocates and research partners are not seen as inclusive methods as such, but as roles in the research process which give people with ID (partial) control over the research. This distinction is adopted in table 3. In round I, agreement was reached on the suitability of all methods and roles to the collaboration approach. In round II, complete agreement disappeared, where clinical research and randomised controlled trials (RCTs) were no longer agreed on as methods allowing for collaboration. The low percentages for the control approach show that no agreement on suitable designs, methods or roles was reached after two rounds.

Table 3

Designs, methods and roles in inclusive health research

	Collaboration		Control
	Round I (%)	Round II (%)	Round I (%)	Round II (%)
Design and research methods
Clinical research	71	65	17	24
RCTs	79	59	21	6
Questionnaires	88	82	38	41
(Semi) structured interviews	88	82	33	41
In-depth interviews	83	88	21	53
Focus groups	92	82	33	53
Roles
Advisory board	79	77	38	35
Self-advocates	71	77	58	53
Research partners	88	100	42	47

For each design, research method and role percentages per response category where calculated. Percentages ≥70% (in bold) reflect agreement.

Academics' rationales regarding designs and methods for inclusive health research

Researchers should strive for the collaboration approach, as ‘neither experts nor people with an intellectual disability would be expected to independently control the research’ (academic 9). This is reflected in the level of agreement on collaboration in table 3. Quantitative research (ie, clinical research, RCTs, questionnaires) was seen as a consultation method. However, people with ID can collaborate in this type of research when given a role as advisory board member, self-advocate or research partner. In the end, it is the researcher's commitment to the inclusiveness of the study that really matters.

Topic II: characteristics of inclusive health research

Characteristics of inclusive health research in general

Little change in agreement occurred between rounds I and II regarding the importance of general characteristics of inclusive health research. Table 4 shows that most agreement was reached on characteristics within the themes accessibility and facilitation. Within the theme recruitment, the importance of a description of the recruitment process was emphasised in the second round. However, representative sampling was not seen as important in either round. No agreement was reached on characteristics relating to the themes reflection and evaluation.

Academics' rationales regarding characteristics in general

Inclusive health research is demanding in terms of resources and the way in which to include people with ID needs to be carefully considered. This is reflected by the many characteristics agreed on within the themes accessibility and facilitation but is in contrast to the few characteristics agreed on for recruitment. The feasibility of expectations regarding inclusive health research needs to be considered: ‘What is reasonable and realistic?’ (academic 2). Representation of people with moderate-to-severe ID is challenging, and methods to include them should be explored.

Although there was no agreement on the themes reflection and evaluation, academics' rationales reflected the need to gain insight into the process, outcomes and added value of inclusive health research, as it is in an early stage of development: ‘We do not want to each be reinventing the wheel!’ (academic 16). Reflection is key in reaching meaningful inclusion; however, reflections on the inclusive process should not overshadow the actual study.

Characteristics of collaboration and control

Table 5 shows that, within the approaches collaboration and control, agreement was reached mainly for characteristics on atmosphere, relationship, engagement, partnership and power rather than for straightforward characteristics such as remuneration, training and salary. The characteristics ‘open, respectful, and confidential atmosphere’ and ‘training and preparation during participation’ were identified as very important for both approaches, indicating that these are characteristics of inclusive health research in general.

Academics' rationales regarding characteristics of collaboration

Payment and valuing participants' time and expertise are seen as an important characteristic of inclusive health research, even though these are not among the agreed on characteristics in table 5. However, payment of people with ID does not necessarily in itself result in meaningful inclusion. To reach meaningful inclusion, training was seen as important to both people with ID and researchers in preparing both for their roles, rights and responsibilities. In line with table 5, training ‘on the job’ is preferred by most academics, because the context would help people with ID understand research. Researchers need to be aware that people with ID are not required to become trained expert-by-experience. A strong relationship between the research partner with ID and researcher as well as mutual respect is crucial to develop trust and confidence: ‘It is at the heart of inclusive research’ (academic 17).

Academics' rationales regarding characteristics of control

Academics were sceptical regarding the control approach because they found complete control unrealistic for any type of research: ‘You cannot just do it your own way’ (academic 10). However, if one strives for control in (part of) the study, the following aspects are crucial: mutual respect and engagement; salary, which was seen as more important than for collaboration as it emphasises the role of people with ID in research; and training on research methods and ethics. Researchers' commitment to listen to and process the input of people with ID was expected to be most important to achieve control.

Topic III: outcomes of inclusive health research

Table 6 presents expected outcomes of inclusive health research per stakeholder group as proposed by academics in this study during round I and whether academics agreed on their importance in round II. Least agreement was found for outcomes relating to the stakeholders’ research(ers) and society. Across all stakeholder groups, agreement was found for the outcomes: improved health(care) and quality of life; more fulfilled needs; increased confidence; and insight into the lives of people with ID.

Table 6

Outcomes of inclusive health research

Stakeholders	Agreement	No agreement
People with ID	People with ID having a voice People with ID feeling valued Research questions relevant to people with ID Reduction in health disparities among people with ID Improved healthcare for people with ID Empowerment of people with ID	Improved communication between people with ID and healthcare professionals Improved health/increased quality of life of people with ID Knowledge translation Involvement of people with ID in the community Increased health literacy Employment of people with ID Acquisition of research skills by people with ID
Research(ers)	Research outcomes suited to the needs of people with ID Research questions relevant to people with ID	Increases research validity Increases knowledge among stakeholders Increases understanding of the lives of people with ID Complete picture New way of thinking among researchers Improved choice in research methods Better understanding of inclusive health research Focus on the transformative process of inclusive research Improved communication within a research team
Healthcare	Most urgent healthcare issues of people with ID Healthcare suited to the needs of people with ID Increased quality of life for people with ID Improved healthcare for people with ID Improved quality and accessibility of healthcare	Mutual understanding between people with ID and healthcare (professionals) Holistic view on healthcare Upskilling of healthcare professionals Knowledge translation Aligns healthcare with the UN Convention on the Rights of Persons with Disabilities
Society	Reduction of health disparities among people with ID Social change Addresses problems relevant to society	Inclusion of people with ID in society Timely and relevant research for society Empowerment of people with ID Holistic view on people with ID Society respecting/valuing the contribution of people with ID Cost-effective services Employment security for people with ID

For each outcome, agreement was achieved when: median=5 (category very important) and IQR=0.

ID, intellectual disabilities.

The in-depth interviews provided additional insight into characteristics on which no agreement on their importance was reached, as well as into topics missing from the Delphi study. The analysis of the in-depth interviews resulted in four themes that can be categorised as nuance, characteristics, operationalisation and added value.

Nuance is needed as to where inclusive health research is appropriate and necessary. Not all people with ID are able and/or want to contribute to research. They need experience with the research topic. ‘I think what we have to be open to is that there are certain research questions and certain decisions in the light of research that are better done with an inclusive research paradigm. But that is not to prescribe it as the sole paradigm as to move forward’ (academic 1).

The importance of the characteristics, ‘payment of people with ID’ and ‘informed shared decisions’ was stressed, even though no agreement on their importance was reached in the Delphi questionnaires. Despite their absence from the Delphi questionnaires, the characteristics ‘continuity of the inclusive process’ and ‘support’ were identified by interviewees as important. Interviewees also emphasised the importance of accessible research material as identified in the Delphi study and stressed that materials should be developed in collaboration with people with ID.

The need for operationalisation, transparency and evaluation of inclusive research was elaborated on. Inclusive health research is seen as a complex and situational process, and there is no ‘blueprint’27 for inclusive health research (academic 14). The need for transparency and reflection was voiced without judgement on the quality of the inclusive health research. Academics envisioned a tool, checklist or guideline providing guidance and support during the inclusive process and showing added value. This tool should support researchers with decisions on ownership and needed adaptations. According to academic 1, the tool should acknowledge ‘that certain elements may not be relevant to their particular question or their particular methodology that they're going to embark upon’. The tool can be used to develop an inclusive research design as well as to reflect on its process: ‘I would like to see some agreement on what should be documented in scientific papers to demonstrate and justify that it was inclusive research’ (academic 4). The tool should establish a common definition of inclusive health research, so that ‘we talk about the same thing’ (academic 4). On the basis of the interviews, the tool should pay attention to the conducting phase, as well as to the planning, disseminating and follow-up phases of inclusive health research. People with ID should be involved in the development of the tool.

Discussion

The aim of this Delphi study—to gain agreement among academics regarding (1) designs and methods, (2) most important characteristics and (3) outcomes of inclusive research—was partially achieved. Academics agreed on (1) collaboration as the most applicable approach to inclusive health research, (2) the high importance of various characteristics regarding the accessibility and facilitation of inclusive health research and (3) the high importance of several outcomes of inclusive health research for people with ID and healthcare. It was challenging to reach agreement on the other topics, and academics indicated that this was due to the complex and context-dependent nature of inclusive health research. The additional in-depth interviews proved helpful in gaining more insight into these topics, such as the development of a tool for inclusive health research. This Delphi study addresses the need for specification of involvement of specific patient groups. Additionally, it makes a comparison with participatory research in general and addresses future research purposes for both inclusive health research and participatory research.

Although a Delphi study design is appropriate to gain agreement among academics, this proved challenging in this study. The dropout rate of 29%, academics' rationales and need for telephone follow-ups indicated that academics found the questionnaires too restrictive. Responding to academics' concerns, this study continued with in-depth interviews to thoroughly question the academics. It was successful in gaining in-depth information on the complexities surrounding inclusive health research and in providing more insight into topics to be explored in future research. Effort was made to collaborate with co-researchers with ID in this study; however, this proved to be challenging. For example, questions were raised how co-researchers could be involved meaningfully in a Delphi study. Co-researchers' perspectives on these and other challenges will be published in a reflection paper in order to try and add to the growing knowledge base of inclusive (health) research. To the best of our knowledge, this Delphi study is the first to approach scientifically, from an academic’s perspective, patients' involvement in health research, and it contributes to the next step in the patient revolution.

It is uncertain whether experiences with participatory research can be translated into inclusive health research. However, this study has found similarities with regard to (1) power distributions and (2) research designs. First, power distribution is a well-discussed topic in participatory research28 and could prove helpful in the inclusive research debate. The Social Model of Disability has been of great influence on participatory research and recognises cultural and social aspects of power relations, which could conceptualise inclusive research further.29 Academics in this study agreed that inclusive health research was a collaborative effort; this corresponds to other inclusive research which states that researchers with and without ID should share control over research in order to collaborate meaningfully.30 This indicates that power distributions should be equal. However, the majority of power in research lies with academics. This cannot be rectified without unravelling the existing power distribution and its underlying structures; this is in line with demands within participatory research.31 Second, participatory research recognises the full range of research designs while collaborating with patients.28 Academics in this study agreed that collaboration can be achieved by means of (semi)structured interviews, in-depth interviews and focus groups, indicating that qualitative designs offer more opportunities than quantitative designs for inclusive health research. This is in line with findings by other inclusive researchers.32 However, academics in this study stated that quantitative designs such as trials and clinical research can be inclusive by offering people with ID the role of advisory board member, self-advocate or research partner. Quantitative research designs might require more effort to be inclusive. Nevertheless, all study designs can be inclusive if necessary. Thus, in this study, the full range of research designs was recognised as a means to collaborate with patients, similar to participatory research. These designs need a rationale for taking on a specific design33 and specification in order to develop inclusive research in the future.34

No agreement was reached on characteristics relating to the themes reflection and evaluation. Nonetheless, in rationales and in-depth interviews, academics strongly emphasised the need for a tool to provide transparency, facilitation and evaluation of inclusive health research and its results. At the same time, academics warned against making judgements about what constitutes ‘good quality’ in inclusive research. Despite the reluctance to make value judgements, the quality and added value of inclusive health research should be studied in order to justify collaboration with people with ID in research. This calls for a critical view on inclusive health research in order to make it work in practice.35 The current absence of a tool might be explained by the developmental phase in which inclusive health research is currently located and by the observed sensitivities with regard to value judgements on the quality of such research.18 Agreement was reached on the importance of relational characteristics such as atmosphere, relationship, engagement, partnership and power, which might complicate objective operationalisation. The aforementioned contradiction between the need for a tool and the sensitivities regarding value judgements underlines the fact that inclusive health research is entangled with ethical values and contextual complexities. Future research should explore inclusive health research in its context in order to study its situationality and what this would imply for developing a tool. Several toolkits and handbooks have been developed for participatory research in general. However, the majority are based on experiences with specific groups such as children and young people36 and people living in developing countries.37 38 We expect the broad terms for inclusive health research and participatory research, such as the aforementioned power distributions and research designs, to be similar. However, adaptations might be needed for detailed characteristics, to adjust for the specific needs of people with ID. In recent years, a number of studies have been published on the needs of this population specifically, which would provide a useful guidance when developing a tool. Examples are: Northway et al's39 paper on reasonable adjustments to promote meaningful participation of people with ID, a study by Nind et al40 on capacity building of people with ID working in research, and an article by McDonald and Stack41 that identifies success factors and challenges for community-based participatory research with people with ID.

Academics agreed on suitable research outcomes and relevant research questions as outcomes of inclusive health research for stakeholder group research(ers), suggesting that there is little added value of inclusive health research for research(ers). The aforementioned outcomes are in line with an earlier literature review on the added value of inclusive health research for research(ers),11 and overlap was found with expected outcomes of participatory research in general. Additionally, for the outcomes of participatory research, the emergence of new research questions and building on the sustainability of research outcomes were identified.42 Earlier inclusive research identified minimising researcher bias,32 increased quality of research for research(ers),11 32 and a means to new knowledge30 as outcomes of inclusive health research. To the best of our knowledge, outcomes of participatory research and inclusive research have not yet been scientifically studied. Academics indicated that nuancing is needed as to where inclusive research is of added value. However, research indicates that one should have to argue for exclusion of people with ID instead of ‘having to argue for their inclusion’.32 Thus, researchers should be made aware of the possible added value of inclusive health research and be nuanced about where the involvement of people with ID has no added value. Future research should explore when and how inclusive health research is conducted in the best way possible. Keeping in mind the specific context in order to approach the patient revolution scientifically is essential.