Development of a research agenda for general practice based on knowledge gaps identified in Dutch guidelines and input from 48 stakeholders.

BACKGROUND: Several funding organizations using different agendas support research in general practice. Topic selection and prioritization are often not coordinated, which may lead to duplication and research waste.
OBJECTIVES: To develop systematically a national research agenda for general practice involving general practitioners, researchers, patients and other relevant stakeholders in healthcare.
METHODS: We reviewed knowledge gaps from 90 Dutch general practice guidelines and formulated research questions based on these gaps. In addition, we asked 96 healthcare stakeholders to add research questions relevant for general practice. All research questions were prioritized by practising general practitioners in an online survey (n = 232) and by participants of an invitational conference including general practitioners (n = 48) and representatives of other stakeholders in healthcare (n = 16), e.g. patient organizations and medical specialists.
RESULTS: We identified 787 research questions. These were categorized in two ways: according to the chapters of the International Classification for Primary Care (ICPC) and in 12 themes such as common conditions, person-centred care and patient education, collaboration and organization of care. The prioritizing procedure resulted in top 10 lists of research questions for each ICPC chapter and each theme.
CONCLUSION: The process resulted in a widely supported National Research Agenda for General Practice. We encourage both researchers and funding organizations to use this agenda to focus their research on the most relevant issues in general practice and to generate new evidence for the next generation of guidelines and the future of general practice.

KEY MESSAGES

The National Research Agenda for General Practice, including 230 prioritized research questions, provides opportunities to encourage research in general practice on topics that really deserve attention.

The agenda can be used by research funding organizations, policymakers and patient organizations as a tool for coordinating and facilitating general practice research.

Introduction

The last two decades scientific research in primary care has expanded [1]. A structural problem, however, is that topic selection and prioritization are often not coordinated, which may lead to duplication and research waste. Most research funds focus on specific clinical areas and specialist medical issues, leaving research questions related to prevailing conditions in general practice unanswered [2-5]. These knowledge gaps are particularly revealed during the process of guideline development when the review of the literature does not provide relevant evidence for primary care.

The Dutch College of General Practitioners (NHG) leads a guideline programme including over 100 evidence-based guidelines covering about 80% of health problems presented in general practice [6]. These guidelines are regularly updated and modified based on recent evidence. During the guideline development process, knowledge gaps are systematically identified and included in an online database. This database offers a unique opportunity to develop a comprehensive research agenda for general practice, to encourage research and funding organizations to design projects and programmes addressing the most compelling questions and issues in general practice. Although this agenda has been developed within the Dutch healthcare system, it may be useful for setting up general practice research programmes in other European countries and for enhancing international collaboration.

In 2015 the NHG took the initiative to develop a research agenda systematically based on knowledge gaps identified in guidelines and input from relevant stakeholders in the Netherlands, in close collaboration with general practice research departments. The project aimed to achieve a nationwide consensus on prioritized research questions within the broad field of GP research. In this article, we describe the development process and a summary of the results.

Methods

Our study started with collecting research questions relevant for general practice. Subsequently, the questions were categorized in International Classification for Primary Care (ICPC) chapters and overarching themes. Finally, the questions were prioritized in an online survey among practising general practitioners (GPs), followed by a national meeting. An advisory board supported the whole process. Table 1 presents an overview of methodological steps in developing the national research agenda.

Table 1.

Methodological steps in developing a national general practice research agenda.

What	Who
1. Compose Advisory board representing scientific GP organizations and university departments	Staff of Scientific Society of GPs
2. Identify relevant stakeholders in GP research	Staff of Scientific Society of GPs and advisory board
3. Identify knowledge gaps and research questions from GP guidelines	Guideline development working groups
4. Ask stakeholders for additional knowledge gaps and research questions	Staff of Scientific Society of GPs
5. Classify knowledge gaps and research questions using ICPC and other relevant dimensions	Advisory board
6. Conduct survey among practicing GPs to prioritize research gaps and research questions	Staff of Scientific Society of GPs
7. Organize national prioritization meeting with GP researchers and other stakeholders	Staff of Scientific Society of GPs
8. Achieve consensus about top 10s and final agenda	Advisory board
9. Endorse final agenda by national scientific society	Board of Scientific Society of GPs
10. Publish and promote agenda among research funds	Scientific Society of GPs and advisory board members

Composition advisory board

Members were selected from: (1) the national working group of heads of the eight research departments of general practice of the University Medical Centres (IOH-R); (2) the working group Science of the National Organization of General Practitioner Trainees (LOVAH); (3) the NHG Member Council; and (4) the NHG Committee for Scientific Research (CWO). The advisory board was supported by the Knowledge Institute of the Federation of Medical Specialists, which has extensive experience with developing research agendas for medical specialists, The Netherlands Patient Federation provided a patient representative for input from the patients’ perspective. The advisory board was chaired by AK, professor of research in general practice and former chair of the Health Council of the Netherlands and the Scientific Council for Government Policy, and supported by three NHG scientific staff members (DK, JW, JvB). The board members participated as independent individual experts, without burden and consultation. All board members completed a form declaring potential competing interests. No relevant competing interests were recorded. The board advised on identification of stakeholders and on classification and prioritization of research questions. They had eight face-to-face meetings and communicated in between by e-mail. Decisions were made based on consensus being aware of arbitrary choices.

Stakeholder involvement

All parties interested in high-quality scientific research in general practice were considered stakeholders. These include boards and groups related to the NHG, the eight research departments of general practice, primary care research institutes, scientific societies in primary and secondary care, organizations funding research, patient and consumer organizations, and governmental agencies relevant to general practice. In total, 96 stakeholders were identified and approached (Table 2).

Table 2.

Stakeholders identified and approached.

Stakeholder	Number
NHG related boards	3
NHG related expert groups	10
General practice research departments	8
National Organization of General Practitioner Trainees	1
General practice collaborative partners (e.g. Dutch Association of GPs)	4
Primary care research institutes	6
Organizations funding healthcare research	24
Scientific societies in primary care	9
Scientific societies in secondary care	14
Patient and consumer organizations	11
Governmental agencies	3
Other (e.g. health insurance organizations)	3

Collecting research questions

Knowledge gaps were derived from all current NHG guidelines (n = 90). Identification of the most essential knowledge gaps with a maximum of five is routinely part of the guideline development procedure. All knowledge gaps are included in an online overview (www.nhg.org/lacunes in Dutch). As a first step for the research agenda, the knowledge gaps from the online overview up to September 2016 were included. In addition, other guidance products of the NHG (n = 5) were screened for research recommendations.

In addition to the knowledge gaps identified in NHG guidelines and guidance products, we asked all stakeholders to provide research questions relevant for general practice. We defined general practice research as research on topics relevant to care provided by GPs, whether or not in collaboration with other healthcare providers. Topics could include somatic, psychological, and social aspects relevant to patient care, organization of care, and education in general practice.

The research questions were categorized according to ICPC chapters. To improve the accessibility of the questions, the advisory group also designed a thematic categorization including 12 research themes, such as common conditions, person-centred care and patient education, collaboration and organization of care, prevention and screening, elderly care and multimorbidity, and infectious diseases. All research questions were grouped according to ICPC chapter and research themes.

Prioritizing research questions

As the first step, an online survey was conducted among members of the NHG. A questionnaire was sent to a random sample of 1700 GPs asking to rate a set of knowledge gaps within one ICPC chapter taking into account the following issues:

How often does the problem occur?

What is the importance of the problem?

Does answering the question help you in your daily GP practice?

They could provide a low (1), medium (2) or high (3) score to each knowledge gap. The average score per item determined the order of priority assessed by the GPs. They were also asked to prioritize 5 out of 12 research themes.

Using the scores of the GPs, couples of members of the advisory group ranked the items grouped within one research theme resulting in top 20 lists of items.

The second step was a national invitational prioritization meeting on October 10, 2017 with representatives of all relevant stakeholders. The aim was to prioritize further all items per ICPC chapter and per theme, resulting in a top 10 list for each of these. Pre-specified criteria to consider during the prioritization process were the burden of disease (for individual patient and society); urgency of the problem; potential impact of research findings; feasibility of research; and implementability of the research findings. Subgroups of about eight participants worked on one ICPC chapter each and they were asked to select a maximum of 10 items. All participants then prioritized the resulting lists. Each participant could select three items per ICPC chapter and three items per theme.

In a third step, the outcomes of the prioritization meeting were discussed by the advisory board and further refined. The NHG board endorsed the resulting research agenda on November 23, 2017.

Results

In total, 787 research questions were collected (Table 3). Of the 96 stakeholders that were approached, 53 (55%) responded, 48 of them providing research questions. Patient organizations submitted 46 research questions. The response of the online survey was 13.6% (232/1700). The national prioritization meeting was attended by 48 GPs involved in research, four non-GP researchers and 16 representatives of other stakeholders, including patient organizations, medical specialists, researchers, and funding organizations.

Table 3.

Number of research questions by source.

Source	Number of research questions
NHG guidelines, n = 90	450
Other NHG guidance, n = 5	15
Input from stakeholders, n = 48	302
Input from GPs (respondents of online survey, n = 232)	20
Total	787

Tables 4 and 5 present the number of research questions related to ICPC chapters and research themes, respectively. Most research questions were medical questions (n = 580). Medical questions were related to therapy (n = 387), diagnosis (n = 177), prognosis (n = 54), prevention (n = 49), and/or screening (n = 47). Other questions were related to quality of care (n = 106), person-centred care (n = 81) and organization of care (n = 52).

Table 4.

Number of research questions related to ICPC chapter.

ICPC chapter (number of NHG guidelines)	Number of research questionsa
A. General and unspecified (5)	74
B. Blood, blood forming organs and immune system (1)	4
D. Digestive (9)	60
F. Eye (2)	13
H. Ear (4)	16
K. Cardiovascular (7)	63
L. Musculoskeletal (9)	96
N. Neurological (6)	34
P. Psychological (8)	84
R. Respiratory (5)	58
S. Skin (16)	75
T. Endocrine, metabolic and nutritional (3)	41
U. Urological (4)	17
WXY. Pregnancy, childbearing, family planning, female genital, male genital (11)	70
Z. Social	7
Miscellaneous	107

Questions could be classified in more than one chapter.

Table 5.

Number of research questions related to research themes.

Research theme	Number of research questionsa
Common conditions	148
Person centred care and patient education	109
Collaboration and organization of care	105
Prevention and screening	92
Mental healthcareb	84
Infectious diseases	83
Elderly care and multimorbidity	66
Youth care	58
E-health and innovation	40
Oncology	30
Diversity	23
Medically unexplained physical symptoms	13
Not applicable	107

Questions could be classified in more than one theme.

This theme is the same as ICPC chapter P.

Nineteen per cent of the questions were related to common diseases, 14% to person-centred care and patient education, 13% to collaboration and organization of care, 12% to prevention and screening and 50% to other themes.

As an example, Box 1 presents the top 10 of research questions related to common conditions, which was the theme that was most often prioritized by the GPs in the online survey. The order was determined at the national meeting, using the priority scores of the GP respondents in the online survey.

Top 10 list of research questions related to common conditions.

What are the short and long-term effects (cure, recurrence, complications) of corticosteroid injections (subacromial and intra-articular) in patients with shoulder complaints in general practice?

Which interventions are effective in functional stomach complaints (= no abnormalities in gastroscopy and no typical reflux complaints present)?

What are risk factors for complications of common infections in primary care?

To what extent is constipation in adults caused by pelvic floor problems?

Are specific recommendations on exercise or an exercise schedule effective in general chronic dizziness?

What is the most effective antibiotic treatment and treatment duration of cystitis in men?

What is the effectiveness of treatment with tape/brace compared to conservative treatment without tape bandage or ankle brace in patients with ankle ligament injury?

What are useful criteria for determining dehydration in older people over the age of 70?

What is the effectiveness of fibre in the treatment of constipation in children and adults?

What is the effect of (frequent) prescription of antibiotics in children on their immune system (especially specific antibodies)?

Discussion

We systematically developed the first national comprehensive research agenda for general practice, including almost 800 research questions covering a wide range of conditions and themes. These questions are practice centred and reflect the core values of general practice, such as integrated, person-centred care and continuity of care [7,8]. The research questions have been classified according to ICPC chapters and themes, offering two entries for searching items. In addition, 23 top 10 lists have been developed to enhance the usability of the agenda for various funding organizations, which are often organized according to specific themes. The project report and overall list of research questions (in Dutch) are accessible on www.nhg.org/onderzoeksagenda.

The National Research Agenda for General Practice fits in a broader national and international development. In the Netherlands, several initiatives have led to similar types of agendas, such as the ‘Dutch National Research Agenda’ (2015), the ‘Research Agenda for sustainable health’ [9], and—more focused on prevention and care—‘Research that makes you better; a reorientation on university medical centre research’ [10]. In addition, there are also knowledge agendas of various scientific societies of medical specialists and research funding agencies. In 2009, the Research Agenda for General Practice/Family Medicine and Primary Healthcare in Europe was published, prepared by the European General Practice Research Network (EGPRN). This agenda partly focused on the core competencies of the general practitioner profession, as defined by WONCA, and no other stakeholders outside the field of primary care were involved in formulating the agenda [11,12].

The benefit of our agenda is the use of specifically formulated knowledge gaps and questions that are important for daily general practice, and the provision of broader research subjects that are important for the future of general practice. In addition, practising GPs and several relevant stakeholders, including patients, were involved in all stages of the process.

A limitation in the process was the response of only 13.6% by the sample of NHG members in the online survey, which may have introduced selection bias. However, such a percentage is standard in (email) surveys among GPs, who are busy and not always motivated for a survey on scientific research. It may be expected that responders were interested in research and were able to provide unbiased input. In addition, their input was further weighed in the stakeholders meeting. Moreover, the absolute number of general practitioners who responded is substantial (n = 232). Their input has been very valuable for the development of this research agenda.

Another limitation is that the agenda has been developed in only one country. Each country has its own social, cultural and economic context, which influences the input and outcome of a research agenda. However, most research questions were derived from analyses of international literature and will be shared with researchers internationally. Next steps are to match the agenda with other research agendas in primary care and to achieve international consensus on a global agenda with the aim of preventing duplication of efforts.

Finally, although the topics included in the research agenda deserve priority, actual developments in science and society may imply that questions not yet included could also be relevant in the near future. In this regard, the research agenda can provide guidance, inviting researchers to substantiate explicitly new developments. The agenda is not static but flexible in accepting newly emerging research questions.

The National Research Agenda for General Practice is now ready for use by research funding organizations, policy makers, and patient organizations. It can be used as a tool for coordinating and facilitating Dutch GP research. Research funders who develop research programmes—aimed explicitly at general practice—can benefit from the questions and topics that emerge from the agenda. They can also use the agenda when considering the practical relevance of submitted proposals. For researchers, the agenda can be helpful in selecting topics for new research projects. In supporting the relevance of the topic, they can refer to the research agenda in addition to their assessment of the importance of the research questions. The research agenda can also make a difference in the funding policy of research institutions and government, for example in the context of initiatives that further strengthen the scientific evidence supporting primary care. Such initiatives are of great importance given the rapid development of knowledge and the increasingly higher demands that society places on primary healthcare. Finally, the research agenda can be used by patient organizations, for instance, when they are involved in decision making about research programmes or projects.

The NHG has an implementation plan including different strategies to keep the agenda alive. It will use its corporate website to encourage communication on ongoing research and adding new research questions. The college has planned meetings with the Dutch government, individual research funds and other stakeholders to support the agenda and to encourage general practice research. Together with ZonMw (The Netherlands Organisation for Health Research and Development), it will also organize a symposium on GP research methodology and aims to repeat this annually.

An English translation of the report including all top 10 lists is available on www.nhg.org/dutch-college-general-practitioners. This provides the opportunity to share research questions and to collaborate internationally on further prioritization. Networks such as the European General Practice Research Network could adopt the methodology and foster collaboration on an international research agenda. Collaborative research projects could be launched to address highly prioritized questions, raising the likelihood of national and European funding.

Conclusion

The Dutch National Research Agenda for General Practice provides a unique opportunity to encourage research in general practice on topics that deserve attention. The top 10 lists can help in selecting the most relevant questions. The agenda can be considered as a rich source that can inspire anyone interested in general practice research. It is, however, never finished. New developments in science, healthcare and society may lead to new research questions and changing priorities, which should lead to ‘a living research agenda’. An international collaborative approach could enhance its richness and application.