Relationship Between Research Culture and Research Activity of Medical Doctors: A Survey and Audit.

PURPOSE: To describe the research capacity and culture, and research activity (publications and new projects) of medical doctors across a health service and determine if the research activity of specialty groups correlated with their self-reported "team" level research capacity and culture.
METHODS: Cross-sectional, observational survey and audit of medical doctors at a tertiary health service in Queensland. The Research Capacity and Culture (RCC) validated survey was used to measure self-reported research capacity/culture at organisation, team and individual levels, and presence of barriers and facilitators to research. An audit of publications and ethically approved research projects was used to determine research activity.
RESULTS: Approximately, 10% of medical doctors completed the survey (n= 124). Overall, median scores on the RCC were 5 out of 10 for organisational level, 5.5 for specialty level, and 6 for individual level capacity and culture; however, specialty-level scores varied significantly between specialty groups (range 3.1-7.8). Over 80% of participants reported lack of time and other work roles taking priority as barriers to research. One project was commenced per year for every 12.5 doctors employed in the health service, and one article was published for every 7.5. There was a positive association between a team's number of publications and projects and their self-reported research capacity and culture on the RCC. This association was stronger for publications.
CONCLUSION: Health service research capacity building interventions may need a tailored approach for different specialty teams to accommodate for varying baselines of capacity and activity. When evaluating these initiatives, a combination of research activity and subjective self-report measures may be complementary.

Introduction

Clinicians play a key role in shaping research agendas, generating research questions, and conducting research that enhances rapid translation of findings.1 Studies have found that health services whose clinicians conduct more research tend to have lower mortality rates, greater organizational efficiency, better staff retention and higher patient and staff satisfaction.2–4 Additionally, a key recommendation to improve the estimated 85% avoidable waste in health research is the increased involvement of health service-embedded clinicians in driving research agendas.1,5

In Australia, building research capacity of clinicians embedded within healthcare services has been identified as a key priority for health research,6 and most recently as one of the 12 priorities for the $20 billion Medical Research Future Fund.7 However, with increasing pressure on public health systems to meet activity targets and provide services to a growing and aging population, finding the time, money and resources to conduct research is challenging. Medical doctors are the second-largest health profession in Australia,8 and have an important role in many types of health research from bench to bedside. However, only 7% of the medical workforce report active involvement in research, and it has been argued this is decreasing.9

A key facilitator of building research capacity is an in-depth understanding of context-specific barriers and facilitators, research culture, and the levels of and drivers for current research activity. In health services in particular, arguments have been made that traditional research output measures, such as publications, may need to be replaced or supplemented by process measures.10 However, there have been very few studies comparing different forms of measurement in this context.

In Australia, the Research Capacity and Culture (RCC) survey has frequently been used to understand the current level of research culture and engagement of a profession, most commonly in Allied Health,11–13 but also in medical professions.14–16 Studies using this tool have found differences in research culture between teams, and findings have been used to inform tailored development of research capacity building strategies.11,13 One study in Allied Health also found that there was no association between a team’s RCC score and their research activity; however, publication and project outputs were low (0–4 publications and 0–8 new projects per team), which limited interpretation.13

In line with the increasing focus on improving health professional research capacity, there is a need to robustly measure and evaluate outcomes of research capacity building initiatives. There is currently little understanding of the relationship between validated measures of research culture and capacity, and traditional research activity and output measures.

Thus, this study aimed to:

describe the research capacity and culture of medical doctors across a health service using the Research Capacity and Culture tool

describe the research activity, in terms of publications and projects, of medical doctors across a health service

determine whether “team” level research capacity and culture on the Research Capacity and Culture tool is significantly different between specialty groups, and whether this score is associated with actual research activity

Materials and Methods

This cross-sectional, observational study collected data using a survey and audit. Ethics approval was obtained from the Gold Coast Hospital and Health Service Human Research Ethics Committee (HREC/10/QGC/177).

Context

Gold Coast Health (GCH) is a publicly funded tertiary health service located in South-East Queensland, Australia. The service includes two hospital facilities of 750 and 403 beds each, as well as outpatient and community-based services. Research is not currently routinely included in medical role descriptions, and engagement in research is variable across the organisation. GCH’s 2019–2022 research strategy has a focus on growing clinician research capacity and establishing a sustainable research culture.17 Strategic incentives and support for research include an annual grant scheme; access to small-scale grants for conference presentations and open access journal fees; an annual research week; a centralised Clinical Trials Unit; and a Research Council and Research Subcommittee of the Board to oversee implementation of the research strategy.17 Considerable work has also been done to increase Allied Health research engagement in the organisation,18 with the RCC tool used across multiple years to measure improvements.13,19 This current study aims to build on this success in the medical stream.

Sample

Convenience sampling was used for the RCC survey. All medical doctors employed by GCH were invited to participate. “Medical doctors” included all professions requiring a medical degree (MBBS, MD or equivalent), including physicians, surgeons, anesthetists, radiologists and others. Data for the audit were gathered from institutional records on publications and projects.

Survey Tool

The Research Capacity and Culture (RCC) tool is a validated questionnaire, which measures indicators of research capacity and culture across three levels – organization, team and individual.8 The tool includes 52 questions (items) on self-reported success or skill in research, including 18 at the organisational level, 19 at the team level and 15 at the individual level. Each item is scored on a scale 1–10, with 10 being the highest possible level of skill or success, as well as an “unsure” option. The RCC also asks respondents to select applicable barriers and motivators to participate in research from a list of 18 of each, and prompts them to add additional barriers/motivators if desired. Lastly, the tool includes a standard set of demographic questions.

Small modifications were made to the tool to better fit the target population, and additional demographic questions were included (eg participant’s facility within GCH). For this study, participants were asked to reflect on their Medical College specialty (eg cardiology, general surgery) when answering questions about their “Team”. This clarification was made as participants could have also interpreted “team” to mean their multispeciality or multidisciplinary team. The tool was administered via a secure online platform (Survey Monkey®).

Procedures

Active promotion and recruitment for the survey was in two stages, – the first in January and February 2019, then a break whilst the organization transitioned to an integrated electronic medical record, and then a second promotion in July and August. Potential participants were provided with an electronic participant information sheet and gave voluntary informed consent prior to participation. The survey took respondents approximately 15 minutes to complete.

Two key audits were performed to collect data on the publications and new projects of doctors in the health service for the calendar years 2018 and 2019. Data was collected for both years as, due to the distributed timing of the survey recruitment, it was determined that both years of data would be relevant to doctors’ RCC responses. For the purposes of the audit, new projects were defined as projects, which obtained a health service governance approval to proceed within that year. Health service-maintained research governance approvals and publication databases were used to identify projects and publications involving medical doctors from the health service. Where necessary, original articles were accessed, or authors/investigators were contacted to clarify information. Information on the number of doctors and FTE status was extracted from internal institutional records.

Data Analysis

Quantitative survey data analysis was performed using Stata 15 (College Station, TX, USA). Survey and audit data were analysed descriptively, using frequencies, percentages, medians and interquartile ranges. “Unsure” responses on the RCC were not included in the analyses but the percentage of “unsure” responses is presented for each item. A one-way analysis of variance analysis with post hoc Scheffe tests was used to determine whether there were differences between specialty groups on the mean of all RCC item scores at the “team” level. Linear regression was used to investigate the relationships between these means and the publications and projects per full-time equivalent (FTE) for each specialty group. Analysis of open-ended survey responses was conducted using inductive qualitative content analysis,20 in which core meanings were derived from the text and grouped into themes. This analysis was completed by a single researcher with experience in health-related qualitative research (CB). Themes were discussed with a subset of the team with qualitative expertise (CB, CN, SM) to reach consensus.

Results

Survey: Quantitative Results

In total, 225 participants consented to complete the survey. Of these, 96 incompletely answered the questions and 129 completed the entire survey. Five of these complete responses were excluded as 4 were not doctors (allied health/nursing/midwifery) and one was a duplicate response. In total, data from 124 survey responses were available for analysis, representing 10.1% of the health service’s estimated total medical workforce.

Participant characteristics are shown in Table 1. The sample had almost equal numbers of males (49.2%) and females (46.8%). Consultants were overrepresented in the sample, making up 72.5% of respondents, whereas they make up slightly less than half of doctors in the health service. Conversely, registrars (12.1% of respondents) and junior doctors (12.9%) were under-represented. Research was part of the role description of 34.7% of respondents, and was not for 42.7%, while almost a quarter (22.6%) of respondents were unsure.

Table 1

Demographic Information and Professional Qualifications

Gender	n (%)
Female	57 (46.0)
Male	61 (49.2)
Other/prefer not to disclose	6 (4.8)
Current employment status	n (%)
Full-time permanent	61 (49.2)
Full-time temporary	23 (18.5)
Part-time permanent	28 (22.6)
Part-time temporary	9 (7.3)
Casual	3 (2.4)
No. of years employed as a Doctor	n (%)
Less than 2 years	11 (8.9)
2–5 years	9 (7.3)
6–10 years	13 (10.5)
11–15 years	28 (22.6)
16–20 years	24 (19.4)
20+ years	39 (31.5)
No. of years employed in this health service	n (%)
Less than 2 years	33 (26.6)
2–5 years	29 (23.4)
6–10 years	29 (23.4)
11–15 years	13 (10.5)
16–20 years	15 (12.1)
20+ years	5 (4.0)
Current career stage	n (%)
Consultant	90 (72.5)
Registrar	15 (12.1)
Junior doctor	16 (12.9)
Not specified	3 (2.4)
Master’s or PhD by research	n (%)
None	104 (83.9)
Completed PhD	9 (7.3)
Enrolled in PhD	2 (1.6)
Completed research Master’s degree	7 (5.6)
Enrolled in research Master’s degree	2 (1.6)
Research-related activities are part of role description	n (%)
Yes	43 (34.7)
No	53 (42.7)
Unsure	28 (22.6)

Note: n= number of respondents.

The median score for research capacity and culture on the RCC was 5 at the organization level, 5.5 at the team level, and 6 at the individual level. Higher scores on the 10-point scale indicate a more positive perception of research culture and capacity. There was a high rate of “unsure” responses for the organization (18.6% of all item responses) and team (14.7%) levels, and a low rate for individual level (2.8%). “Unsure” responses relating to external funding, applications for scholarships/degrees, mechanisms for monitoring research quality, consumer involvement, and research software made up over a fifth of responses at both the organisational and team level. At the organisational level only, ensuring the availability of career pathways and having a policy/plan for research development also resulted in over 20% “unsure” responses.

Median scores for each item at the organizational level ranged between 3 and 7 (Table 2). Key strengths of the organisation’s research culture were “promotes clinical practice based on evidence” (median=7), “engages external partners (eg universities) in research” (6), “supports the peer-reviewed publication of research” (6) and “has regular forums to present research findings” (6). Lowest success was reported for “ensures staff career pathways are available in research” (3) and “has funds, equipment or admin to support research” (4).

Table 2

Median Score for Organisation Level RCC Items, Arranged in Descending Order

Item	Median Score	IQR	% Unsure Responses
Promotes clinical practice based on evidence	7	5–8	3.2
Engages external partners (eg universities) in research	6	4–8	12.1
Supports the peer-reviewed publication of research	6	3–8	16.1
Has regular forums to present research findings	6	3–7	10.5
Supports a multidisciplinary approach to research	5	4–7	18.5
Encourages research activities which are relevant to practice	5	4–7	6.5
Accesses external funding for research	5	3–7	25.8
Has resources to support staff research training	5	3–7	11.3
Has identified experts accessible for research advice	5	3–7	16.9
Has a plan or policy for research development	5	3–6	20.2
Has software programs for analysing research data	5	3–7	36.3
Has mechanisms to monitor research quality	5	3–6	32.3
Supports applications for research scholarships or degrees	5	3–6	27.4
Ensures that organisational planning is guided by evidence	5	3–6	16.9
Has executive managers that support research	5	3–7	16.1
Has consumers involved in research	5	2.5–6.5	29.8
Has funds, equipment or admin to support research	4	2–6	13.7
Ensures staff career pathways are available in research	3	2–5	21.8

Abbreviation: IQR, Interquartile range.

Median scores for items at the team level ranged from 3 to 7 (Table 3). Key strengths of team-level research culture reflected the organizational level, including “conducts research activities which are relevant to practice” (7), “supports peer-reviewed publication of research” (7) and “supports a multidisciplinary approach to research” (7). Lowest success was reported for “has funds, equipment or admin to support research” (3) and “has incentives and support for research mentoring activities” (3).

Table 3

Median Scores for Team Level RCC Items, Arranged in Descending Order

Item	Median Score	IQR	% Unsure Responses
Conducts research activities which are relevant to practice	7	5–9	6.5%
Supports peer reviewed publication of research	7	4–9	10.5%
Supports a multidisciplinary approach to research	7	3.5–9	10.5%
Has team leaders that support research	7	3–8	5.6%
Does planning that is guided by evidence	7	4–8	10.5%
Disseminates research results at forums/seminars	6	4–8	9.7%
Has applied for external funding for research	6	3–9	25.8%
Has external partners (eg universities) engaged in research	6	3–8	18.5%
Has identified experts accessible for research advice	6	3–8	14.5%
Provides opportunities to get involved in research	6	3–8	2.4%
Supports applications for research scholarships/degrees	5	2.25–8	21.0%
Has mechanisms to monitor research quality	5	2–7	27.4%
Ensures staff involvement in developing the plan for research development	5	2–7	11.3%
Has resources to support staff research training	5	2–7	9.7%
Has consumer involvement in research activities/planning	4	2–7	25.0%
Does team level planning for research development	4	2–7	12.1%
Has software available to support research activities	4	2–6	28.2%
Has incentives and support for research mentoring activities	3	2–6.5	16.9%
Has funds, equipment or admin to support research	3	2–6	12.9%

Abbreviation: IQR, Interquartile range.

Median scores for items at the individual level ranged from 3.5 to 7 (Table 4). Strengths in individual research success were related to evidence-based practice (EBP) skills, including “finding relevant literature” (7), “integrating research findings into practice” (7) and “critically reviewing the literature” (7). The lowest rated item was “securing research funding” (3.5).

Table 4

Median Scores for Individual Level RCC Items, Arranged in Descending Order

Item	Median Score	IQR	% Unsure Responses
Finding relevant literature	7	7–8	0.8
Integrating research findings into practice	7	6–8	2.4
Critically reviewing the literature	7	6–8	1.6
Collecting data (eg surveys, interviews)	6	5–8	1.6
Writing for publication in peer-reviewed journals	6	4–8	4.8
Writing a research report	6	4–7.25	3.2
Using a computer referencing system (eg Endnote)	6	4–7.25	0.8
Analysing quantitative research data	6	3–7	1.6
Using computer data management systems	6	3–7	2.4
Writing a research protocol	5	4–7	1.6
Writing an ethics application	5	3–7	3.2
Providing advice to less experienced researchers	5	2.5–7	4.0
Designing questionnaires	5	3–7	4.0
Analysing qualitative research data	5	2–7	3.2
Securing research funding	3.5	2–5	6.5

Abbreviation: IQR, Interquartile range.

Table 5 shows the most commonly reported barriers and motivators by percent of total survey respondents. “Lack of time for doing research” and “Other work roles take priority” were the most commonly reported barriers, identified by 83.9% and 82.3% of respondents, respectively. In decreasing order of frequency, other common barriers were the “lack of funds for research” (57.3%), “desire for work/life balance” (45.2%), “lack of a coordinated approach to research” (45.2%), and “lack of skills for research” (45.2%). Only 4% of respondents reported “not interested in research” as a barrier. The most commonly reported motivators were “to develop skills” (70.2%), followed closely by “increased job satisfaction” (66.9%). Other common motivators included “problem identified that needs changing” (58.9%), “career advancement” (58.1%) and “to keep the brain stimulated” (57.3%).

Table 5

Reported Frequency of Personal Barriers and Motivators to Conducting Research, Arranged in Descending Order

Item – Barrier	% of Response Across n=124	Item – Motivator	% of Response Across n=124
Lack of time for doing research	83.9	To develop skills	70.2
Other work roles take priority	82.3	Increased job satisfaction	66.9
Lack of funds for research	57.3	Problem identified that needs changing	58.9
Desire for work/life balance	45.2	Career advancement	58.1
Lack of a co-ordinated approach to research	45.2	To keep the brain stimulated	57.3
Lack of skills for research	45.2	Desire to explore a theory/hunch	53.2
Lack of support from management	39.5	Dedicated time for research	52.4
Lack of software for research	39.5	Links to universities	50.8
Isolation	36.3	Increased credibility	50.0
Lack of access to equipment for research	34.7	Mentors available to supervise	45.2
Other personal commitments	34.7	Opportunities to participate at own level	45.2
Intimidated by research language	21.0	Colleagues doing research	42.7
Different experience levels of team members (applies to team only)	17.7	Research encouraged by managers	37.9
Staff shortages (applies to team only)	16.9	Grant funds	37.1
Intimidated by fear of getting it wrong	14.5	Research written into role description	36.3
Not interested in research	4.0	Forms part of post-graduate study	25.0
Lack of library/internet access	2.4	Team building (applies to specialty group only)	25.0
Other barrier/s (please specify) – lack of dedicated research support roles	4.8	Study or research scholarships available	21.0
Other barrier/s (please specify) – lack of pathways to engage in research	3.2	Required as part of Specialty College Training	20.2
Other barrier/s (please specify)- difficulty with research ethics and governance processes	1.6	Other motivator/s (please specify)	0.0
Other barrier/s (please specify) – lack of value placed on research in organisational culture	1.6
Other barrier/s (please specify) – contract role does not allow research	0.8

Survey: Qualitative Results

Twenty-nine percent (n= 36) of respondents provided free text responses to the question “Do you have any final comments or suggestions about the survey or research in general?”. Responses were a mean of 34 words in length. The four main themes are summarised in Table 6. The most common theme (n=21) was expansion on barriers to doing research, which were mostly reflective of the quantitative results. A significant portion (n=16) described the tension for change,21 emphasising there was a need for improvement in research engagement, positing that the health service was behind others in this aspect, and outlining frustrations with a perceived lack of focus on and support for research in the service. Others (n=11) included suggestions for potential strategies for improving the research culture at GCH in their response. These included suggestions like dedicated time for research, research support staff, making it easier to link in with potential projects, and calls for improved research culture and planning in the health service. A small number of respondents (n=4) outlined their perceptions of the benefits of increased research engagement as justification for the need for change.

Table 6

Codes from Qualitative Analysis of Final Free Text Question

Expanding on barriers (n= 21)Example quote: “It is so very difficult to balance doing the clinical job with finding time to do, or supervise, research, especially when most of the time is spent just getting the project running!”	• Difficulty finding out about research projects
	• Research is not valued by leadership
	• Lack of research/nonclinical time and overloaded clinical rosters
	• Lack of skills and knowledge, including HDR experience
	• Lack of interest in department
	• Lack of general support and infrastructure
	• Ethics and governance process and long start up process for projects
	• Lack of support staff
	• Lack of awareness about what support is available
	• Casual contracts
Potential strategies (n= 11)Example quote: “Please make available projects easier to find. Consider an online noticeboard where all research groups post available roles and projects.”	• Involvement with statewide clinical trials networks
	• More professorial appointments
	• Pathways to make available projects easier to find, including online listing
	• More long-term planning
	• Appoint more research support staff
	• Increase awareness of the benefits of research at an executive level
	• Substantial investment in changing research culture
	• Dedicated time for research, including rostering more clinicians
	• More funding and access to HDRs
	• Research criteria should be included in performance reviews
Benefits of a research culture (n= 4)Example quote: “At exec level there needs to be awareness this investment will be paid back in spades in 1) economic return; 2) job satisfaction; 3) career progression; 4) staff retention.”	• Attraction and retention of high quality trainees
	• Retention of quality staff
	• Economic return
	• Job satisfaction
	• Career progression for staff
	• Generally a good thing for the health service
Tension for change (n= 16)Example quote: “Urgent need for institutional support for formal research ….training among clinical staff in all disciplines”	• General statements that there is a need for change
	• Our health service is not as research active as others in the region
	• Increased research engagement is needed to strengthen our identity as a tertiary institution
	• Despite being part of clinical role descriptions, research is mostly completed in personal time
	• The health service is focused on short-term operational concerns at the expense of research

Project Audit

There were 266 research projects that received health service governance approval from January 2018 to December 2019. 74.1% (n=197) of these had a medical doctor from the health service as an investigator. Data was not collected on investigators who were not affiliated with the health service. For every 12.5 doctors employed in the health service, one project was commenced each year.

Table 7 displays further details on these 197 projects. The health service was the only site in 50.8% of the projects, the lead site of a multisite study in 6.6%, and nonlead site on a multisite study in 42.6%. Most (80.2%) projects involved doctors from a single specialty in the health service, while 19.8% involved doctors from multiple specialties (eg cardiology and rheumatology). A third of projects involved collaboration with other professional streams, such as nursing and Allied Health.

Table 7

Characteristics of Publications and Projects from Medical Doctors in 2018 and 2019

		n (%)
Projects (n= 197)	2018	92
	2019	105
Site information	Health service as only site	100 (50.8%)
	Health service as lead site of a multisite study	13 (6.6%)
	Health service as nonlead site on a multisite study	84 (42.6%)
Inter-specialty collaborationa	Investigators from only one specialty	158 (80.2%)
	Investigators from more than one specialty (eg cardiology and rheumatology)	39 (19.8%)
Inter-professional collaborationa	Investigators only from the medical stream	132 (67%)
	Investigators from nursing and medical streams	28 (14.2%)
	Investigators from Allied Health and medical streams	16 (8.1%)
	Investigators from otherb and medical streams	6 (3%)
	Investigators from more than two streams (eg nursing, Allied Health and medical)	15 (7.6%)
Publications (n= 327)	2018	175
	2019	152
Number of unique medical doctor authors and their number of publications in 2018	2018	161
	1 publication	117 (72.6%)
	2 publications	22 (13.7%)
	3–5 publications	19 (11.8%)
	6–9 publications	1 (0.6%)
	10+ publications	2 (1.2%)
Number of unique medical doctor authors and their number of publications in 2019	2019	162
	1 publication	122 (75.3%)
	2 publications	31 (19.1%)
	3–5 publications	5 (3.1%)
	6–9 publications	2 (1.2%)
	10+ publications	2 (1.2%)
Authorship order	Medical doctor as first or last author	174 (53.2%)
	Medical doctor as middle author	153 (46.8%)
Inter-profession collaborationa	Authors only from the medical stream	282 (86.2%)
	Authors from nursing and medical streams	26 (8.0%)
	Authors from Allied Health and medical streams	14 (4.3%)
	Authors from otherb and medical streams	3 (0.9%)
	Authors from more than two streams (eg nursing, Allied Health and medical)	2 (0.6%)
Publication type	Primary research	196 (59.9%)
	Case reports	42 (12.8%)
	Systematic reviews, scoping reviews and meta-analyses	19 (5.8%)
	Protocols	4 (1.2%)
	Other article types (eg correspondence, topic summaries, opinions, etc)	66 (20.2%)

Notes: aOnly represents collaborations within the health service, specialty/profession data was not collected for other institutions; bOther refers to staff outside of medical, nursing or Allied Health streams (eg statisticians, human resources professionals).

Publication Audit

There were 479 total publications that included health service staff between January 2018 and December 2019. More than two thirds (68.3%; n=327) of these publications had a medical doctor from the health service as an author (data was not collected about authors not employed in the health service). For every 7.5 doctors employed in the health service, one article was published each year.

Table 7 displays further details on these 327 publications. There were around 160 unique authors each year (approximately 13% of GCH doctors), and just over a quarter of these published in both years. In 2018, 72.6% published a single paper, 13.7% published two papers, and only 1.2% published over ten papers. These results were similar in 2019.

A medical doctor from the health service was first or last author in 53.2% of the publications, and 13.8% involved authors from another professional stream in the health service, such as nursing or Allied Health. The most common publication type was primary research (59.9%), followed by other article types like letters or opinions (20.2%) and case reports (12.8%).

Relationship Between Survey Results and Research Activity for Specialty Groups

“Team” level scores on the RCC were summarised into 7 broad specialty groups (Table 8), which were (in no particular order): emergency medicine; surgery and intensive care; anaesthetics; psychiatry; paediatrics, obstetrics and gynecology; and 2 physician groups split according to which division of the health service the specialty was placed. One response could not be categorised as a specialty group. Specialty groups were anonymized as Groups 1–7 in accordance with this study’s ethical approval.

Table 8

Mean Scores for Team Level RCC Items and Publication and Projects per FTE, Separated by Broad Specialty Group, Arranged in Descending Order

Median Scores	Group 1 (n=16)	Group 2 (n=14)	Group 3 (n=12)	Group 4 (n=17)	Group 5 (n=18)	Group 6 (n=20)	Group 7 (n=26)
Survey results
Overall mean for team-level items	6.4	4.0	4.6	5.2	7.8	3.1	6.0
Overall SD for team-level items	2.1	2.8	2.3	2.1	1.1	2.4	1.7
Audit results
Number of publications per FTE	0.50	0.01	0.15	0.33	0.67	0.18	0.44
Number of new projects per FTE	0.26	0.08	0.09	0.31	0.21	0.13	0.45

Differences were identified in the mean total-item RCC scores at the “team” level between specialty groups (P<0.0001). Group 5 had the highest mean score (7.8) which was greater than Group 6 (3.1, P<0.0001), Group 2 (4.0, P<0.0001), Group 3 (4.6, P=0.010), and Group 4 (5.2, P=0.036). Group 6 had the lowest mean score (3.1) and was lower than Group 7 (6.0, P=0.003) and Group 1 (6.4, P=0.006) as well as Group 5.

Linear regression showed a relationship between mean “team” level RCC score and both projects (Figure 1) and publications (Figure 2) per FTE. Each additional project or publication per FTE corresponded to an increase of 5.3 (95% CI 2.0–8.6; P=0.002) or 6.6 (95% CI 4.8–8.5; P<0.0001) in the mean “team” level RCC score, respectively. It was estimated that of the total variance in “team” level RCC score, 29.95% was associated with variance in publications per FTE and 7.91% in projects per FTE.

Figure 1

Relationship between Projects per FTE of teams and the teams’ mean RCC scores.

Figure 2

Relationship between Publications per FTE of teams and teams’ mean RCC scores.

Discussion

The research capacity and culture of medical doctors at individual, team and organisational levels was moderate, with medians of 6, 5.5, and 5, respectively. This level of research culture and capacity, and a pattern of team and individual RCC scores being higher than organisational-level scores, is reflected in similar studies including medical doctors in other Australian health services.15,16

The pattern and spread of results between the highest- and lowest-rated items found in this study is also broadly reflected in the results of these other studies.14,15 This includes the general trend of Evidence Based Practice-related items scoring more positively than pure research-related items, and items relating to funding, career pathways and incentives for research activity consistently scoring the lowest.22 This reflects progress towards the commonly cited goal that all clinicians should understand/use research, while fewer will participate in or lead it.23 Lack of time and other work roles taking priority as the most common barriers to undertaking research are also consistent across the literature for clinicians within health services.12–15

A 2017 study using the RCC with Allied Health professionals in the same health service returned scores consistently 1–3 points higher for the organisation level, with an overall median of 7.19 Reasons for this are unclear, as it is unlikely the organisation’s research culture has changed so significantly in that time. The Allied Health clinicians also scored their team-level success slightly higher, with a median of 6, and their individual capability lower, with a median of 5.19 Notably, both studies also found a high level of “unsure” responses, especially at the organization level, implying that the health service could improve communication and promotion of institutional research supports and initiatives.19

Consistent with the Allied Health study, differences were identified between specialty groups at the team level, with mean scores varying from 3.1 to 7.8. Other research has also shown that different teams within the same health service may have different barriers, motivations and levels of research capacity and current activity.11,13,19,24,25 Literature on research capacity building in health settings has found that team-based approaches are likely to be most effective, as they allow strategies to be tailored to specific needs.11,24–26 Tools like the RCC are a useful way for health services to capture their teams’ current research engagement and needs.

This study found that medical doctors were the largest producers of research projects and publications within the health service. Over a third of projects involved collaborations with other professions like Allied Health and nursing, reflecting the importance of the multidisciplinary team in modern models of care.27 Due to differences in how studies measure and report research activity,28 it is difficult to compare research activity with other health services.29,30 Available international literature on research outputs from medical clinicians usually focuses on specific specialties, often further limited to registrars and academic doctors, rather than a whole of health service measurements.16,31–35 However, one self-report survey found that for every 12.8 Australian physiotherapists employed in tertiary facilities, one article was published per year, compared to 7.5 doctors in this study.36

We identified an association between a specialty group’s research activity and their self-reported, subjective research culture and capacity. Publications demonstrated a stronger association than projects, likely because the two highest scoring groups on the RCC (1 and 5) had relatively few projects. Further investigation showed that one of these groups had a high proportion of multiphase, multisite and complex projects. This indicates that the number of projects may be a poorer indicator than the number of publications, as a simple project like a retrospective chart audit is counted equivalently to a multisite, multiphase interventional trial. The same can also be true of publications; however, complex projects will likely result in multiple publications, which helps offset this effect.

Previous research has called into question the utility of research activity and output measures, such as publications for measuring clinician research capacity. As producing research is not the core role of clinicians, an improvement in research culture is likely to have a significant time lag before a measurable increase in outputs is realized.10,12,37,38 Some authors have argued that self-report measures of research capacity and culture should be combined with traditional research activity measures.12,13 However, this is the first study to identify that the two types of measure are associated, adding further weight to this argument.

Limitations

Limitations to the study were the low (10.1%) response rate for the survey and the fact that the sample was not random. Other studies using the RCC have achieved both lower14,15 and higher12,13,16 response rates. Selection bias may also affect results, as it is likely that those interested in research were more likely to respond to the survey. This may also explain the overrepresentation of consultants compared to junior doctors. A poor response rate from junior doctors has been found in other surveys of research culture.24 Due to this, the results may be more likely to reflect the opinions of senior doctors, and should be interpreted with caution for the junior doctor and registrar populations.

Future Directions

Locally, results from this study were fed back to key strategic groups at Gold Coast Health in February 2020; however, the Covid-19 pandemic, staffing changes, and an organisational restructure have slowed the translation of this work. Nevertheless, funding has been obtained for a Knowledge Translation study to identify and implement evidence-based strategies for increasing medical engagement in research. The findings of this current study are being used to inform this work, provide a baseline measure, and help localise and tailor potential strategies.

In terms of wider implications, the findings of this work may help inform the approach to research capacity building in similar settings. This study has demonstrated that research capacity and culture vary widely between teams within an organisation, and comparison with other studies shows that it varies between organisations. While this means that the results should not be directly applied to other settings, there are common patterns, and this work adds to the literature on the research capacity and culture of medical professionals15,16 to provide a set of results for comparison. More importantly, however, there are few studies reporting data on health service-wide research activity, in contrast to the plethora of research describing the research outputs of universities.39 As health services and clinicians are increasingly called upon to be producers, not just consumers, of research,3,4 systematic study of their research activity is essential.

The link between RCC results and actual research activity of teams should also be further explored, especially in longitudinal research. It would be valuable to determine whether there is delay between improvements in a team’s scores on the RCC survey, and translation into increased research activity in the form of new projects. This would serve to delineate whether a self-report measure like the RCC can reflect improvements in research capacity and culture more rapidly, thus being a more sensitive measure to change than traditional research output measures.

Conclusion

This study reinforces the most significant challenge faced in supporting clinical research within a tertiary health context governed by activity-based funding allocations: providing medical staff time to engage in research activities. New approaches to address this in an increasingly constrained fiscal environment are needed. This study also demonstrated significant differences between team’s reported research activity and culture, indicating that research capacity building initiatives may need to be tailored to specialty groups. Objective activity measures, particularly publications, were shown to be associated with a team’s self-rated research capacity and culture. A combination of subjective process and objective activity outcome measures may therefore be complementary when measuring the impact of research capacity building initiatives. The results of the RCC in this study are intended to identify areas for improvement and provide a baseline for multi-faceted and tailored research capacity building programs in the health service.