The beginning of the research stream in family medicine residency program at McMaster University.

BACKGROUND: To examine research background, attitudes, knowledge and skills of family medicine residents with regard to primary care research and to compare residents who elected to participate in the research stream with those who did not.
METHODS: Mailed survey of Family Medicine residents at McMaster University in 1998, 70% (52/74) of whom responded. The main outcome measures consisted of research background; attitudes towards primary care research and research activities during residency program; knowledge and skills in applying it in biostatistics, epidemiology, and research design.
RESULTS: The vast majority of the residents reported previous research experience and/or some training in epidemiology and biostatistics. Residents in the research stream were more likely to be female and were positive towards primary care research: they were more interested in research, more interested in obtaining more research training while a resident, and placed more importance on developing research early in medical education. The research stream residents had stronger views regarding perceived lack of support staff and lack of time for research. There were no statistically significant differences between the research stream and other residents in terms of research knowledge and skills in applying it.
CONCLUSIONS: Attitudes towards research rather than research knowledge or skills seemed to distinguish those selecting to be in our new research stream at the inception.

Background

Family medicine is evolving from a specialty established primarily to meet the demand for skilled clinicians to serve rural and urban communities, to a research based discipline mirroring other specialties whose progress in knowledge is based on methodologically sound research. Without primary care research to advance medical knowledge, many myths and misconceptions in family medicine will undoubtedly persist [1,2]. Unfortunately, there is a shortage of primary care researchers in North America. Only 0.2% of graduates of US family practice residencies surveyed in 1991 were primarily entering research careers [3]. To partially address this shortage of researchers, many family residency programs in US and Canada require residents to complete a research project prior to graduation in the hopes of stimulating their graduates to pursue research.

Although residency program directors may embrace research during residency training [1,4-9] there have been few studies on whether research during residency training increases the number of primary care researchers. The literature suggests that the secret to creating a successful research environment during residency involves: starting early in the residency program, support from the program director or a committed faculty member, allotment of time for research, an integrated and structured research curriculum, use of evidence based medicine, faculty involvement, a research committee, an easily accessible research professional, research visibility, and opportunities for presenting [5,6,10]. Frequently cited barriers to effective research curricula include: lack of protected time, insufficient funds for support staff, lack of mentors and lack of research skills in faculty members [1,2,4,7,8,10-14]. Studies have shown that postgraduate research involvement is strongly associated with research participation during medical school [3,10,14-16]. However, it appears that required research during medical school may not lead to more physician-investigators [16].

The "Family Medicine Research Stream" at McMaster University Postgraduate Program in Family Medicine was established in 1997/98 academic year and was designed to provide the environment needed for interested residents to engage in meaningful research. Our goal was to maximize optimal use of limited research resources by enhancing research experience for a self-selected group of family medicine residents.

The family medicine residency program at McMaster University is one of the handful of programs in North America that does not require its residents to submit a mandatory research or academic project. However, resident research is encouraged and an elective in research is available to those interested. The program has a research director and a full-time research coordinator who both have expertise in research methodology, statistical analysis, knowledge of research application software, and grant-writing experience.

Twelve out of 74 residents who were enrolled in the residency program in 1997/98 academic year joined the research stream. The research stream curriculum, while flexible in scope, provides structured resources for administrative, methodological and supervisory support. Residents are encouraged either to develop their own projects or to link with ongoing research activities within the department or elsewhere. Workshops are provided 3 to 4 times per year for educating residents on research methodology and statistical analysis, discussion of research projects, and brainstorming future ideas. Research Workshops also involve a half-hour plenary session followed by a roundtable discussion in which residents give a progress report on their activities. Residents in the research stream are invited to tap into the expertise of mentors, while addressing any obstacles, challenges, or successes they may be experiencing in their own research projects.

Our principle objective in doing the survey was to describe family medicine residents in terms of their research background, attitudes, knowledge and skills with regard to primary care research and to determine whether there were important differences between residents who chose to participate in the research stream and those who chose to remain in the regular curriculum. We were also anticipating that the survey will provide us with the information required to develop a relevant research curriculum for the family residency research stream.

Methods

Study population

The study sample consisted of family medicine residents attending the Departments of Family Medicine at McMaster University during 1997/98 academic year. All residents were eligible to participate.

Study design and survey procedures

A mailed survey questionnaire was distributed in January 1998 accompanied by a cover letter signed by the family medicine postgraduate director at McMaster University via interhospital mail to all family medicine residents enrolled in the program in the 1997/98 academic year. The follow-up strategies using the modified Dillman method [17] included mailed and emailed reminders, full mailings of the questionnaires, and telephone calls to non-respondents by a research assistant.

Survey instrument

As there was no existing survey instrument, a questionnaire was developed and screened for content by the investigators. While the questionnaire had adequate face and content validity, it was also pilot-tested on a convenience sample of family medicine faculty at McMaster University and then further refined. There were three sections to the self-administered questionnaire totaling four pages. Section 1 obtained information about educational background, past experience in research and biostatistics, and residents' research interest using a set of open-ended questions. Section 2 sought information about residents' attitudes regarding research in family medicine on 15 items assessed on a 10 point Likert scale with 1 being strongly disagree and 10 strongly agree. The last section consisted of 18 multiple-choice questions (MCQ) intended to assess respondents' knowledge and skills in biostatistics, epidemiology, and research design. The questionnaire is available from the principal author upon request.

Analysis

Data analysis was carried out using SPSS for Macintosh software (version 4.0 for Macintosh, SPSS Inc., Chicago, 1990). The descriptive statistics (frequencies, means and SD) were calculated for all questions. The mean percent MCQ score was calculated for the knowledge questions. The significance of differences in response between the research stream and other residents was measured using chi-square tests or Fisher's exact test for categorical variables and a multivariate analysis of variance (MANOVA) omnibus test followed by Student's t-tests for continuous variables. Because of multiple comparisons p < .01 was accepted as the minimum criterion for significance and two tailed tests were used in all analyses.

Results

Response rate and respondents profile

A total of 52 questionnaires, 11 from the residents in the research stream and 41 from the residents in the regular program were completed, resulting in a response rate of 70% (52/74). The response rate was higher for residents in the research stream (91%, 11/12) than for those in the regular curriculum (66%, 41/62) (Fisher's exact test, p = .095). The overall demographic and research background profile of respondents as well as a comparison between the research stream residents and other residents in terms of these characteristics is given in Table 1. Nineteen percent of residents (10/52) reported to have advanced degrees, 71% (37/52) described previous research experience, and 61% (32/52) had some training in epidemiology and biostatistics. There were no statistically significant differences between the research stream and other residents with regard to the above attributes.

Table 1

Demographic profile of respondents, overall and by research stream status

Characteristic	Overall n = 52	Research stream n = 11	Regular stream n = 41	p value
Female: n (%)	32 (62)	10 (91)	22 (54)	.035
McMaster graduate: n (%)	21 (40)	7 (64)	14 (34)	.095
R1'S: n (%)	29 (56)	8 (73)	22 (54)	.319
Advanced degree: n (%)	10 (19)	2 (18)	8 (20)	1.000
Research experience: n (%)	37 (71)	8 (73)	29 (71)	1.000
Training in biostatistics or epidemiology: n (%)	32 (62)	8 (73)	24 (59)	.497
The most important factors that would facilitate research activity:
More time	18 (35)	3 (27)	15 (37)
More support	22 (42)	8 (73)	14 (34)
Not interested	12 (23)	0 (0)	12 (29)	.039*

* Chi-square test, all other comparisons used Fisher's exact test

The respondents indicated that broadly defined research support (42%) and adequate time (35%) were the two most important factors that would facilitate greater research interest and activity during family practice residency while 23% indicated no interest in research. The research support category included factors such as increased research mentorship, research opportunities, role modeling, biostatistical support and funding. While the research stream residents were more likely to be female and graduates of McMaster University, these differences were not statistically significant at p < .01. The participants were also asked to list their three main research or interest areas. The research stream residents listed more key words than those in the regular program did; however, the most frequent responses in both groups were similar and most frequently included health promotion/public health, preventive medicine, and adolescent health.

Attitudes

The residents' general attitudes towards the primary care research and research activities during residency program were assessed through 15 items presented in Table 2. Statements that received highest mean scores, either in terms of agreement or disagreement, included a strong endorsement for a statement advocating strengthening of the scientific foundations of clinical practice (mean = 7.9, SD = 1.7), agreement with a statement that there is a lack of time to pursue research during the residency training (mean = 7.4, SD = 2.0), and strong disagreement with statements that there are no opportunities to participate in primary care research (mean = 3.6, SD = 1.9) and that there are no family medicine research role models in our department (mean = 3.9, SD = 2.3). The remaining 11 statements received mean scores clustered around the middle of the scale.

Table 2

Attitudes of family practice residents towards primary care research and research activities during residency training

STATEMENT (all items measured on 10 point Likert scale: 1 'Strongly disagree'; 10 'Strongly agree)	Overall Mean (SD) (n = 52)	Research stream Mean (SD) (n = 11)	Regular stream Mean (SD) (n = 41)	p* value
With the increased emphasis on primary care, there is a need for strengthening the scientific foundations of clinical practice.	7.9 (1.7)	9.1 (1.5)	7.6 (1.6)	.008
Family medicine residents should be required to carry out research projects as part of their residency.	4.6 (2.8)	6.2 (3.1)	4.2 (2.6)	.037
Research is adequately emphasized in family medicine training at McMaster.	5.6 (2.0)	4.0 (2.0)	6.0 (1.8)	.002
The lack of a critical mass of investigators is a major impediment to the development of primary care research.	6.1 (1.9)	7.5 (2.4)	5.7 (1.6)	.007
Developing a culture of primary care research must occur early in professional medical education.	6.6 (2.4)	8.5 (2.3)	6.1 (2.2)	.003
It is imperative that the "evidence" in the Evidence-Based Medicine paradigm is developed by primary care researchers.	5.8 (2.3)	5.9 (1.9)	5.8 (2.4)	.845
Every family medicine resident should be assigned to a research mentor at the beginning of their training.	5.2 (2.9)	6.0 (2.9)	5.0 (2.9)	.301
Family medicine faculty at McMaster are knowledgeable about research.	6.3 (1.8)	5.4 (2.0)	6.5 (1.6)	.052
I am not interested in research activities.	5.5 (3.3)	2.2 (2.1)	6.5 (2.9)	.000
There is a lack of dedicated time to pursue research activities during the residency training.	7.4 (2.0)	8.3 (1.7)	7.2 (2.1)	.121
I have good research skills.	5.4 (1.9)	5.7 (2.6)	5.3 (1.7)	.549
I would like to get more research training as part of the family medicine residency training.	5.9 (3.1)	8.8 (2.0)	5.1 (2.8)	.000
There are no opportunities for family medicine residents to participate in primary care research.	3.6 (1.9)	3.5 (1.6)	3.6 (1.9)	.838
There are no family medicine research role models in our department.	3.9 (2.3)	3.8 (2.1)	3.9 (2.4)	.892
There is sufficient statistical/epidemiological support available to residents who are interested in research.	5.8 (2.1)	4.5 (2.0)	6.2 (2.1)	.017

* Based on Student's t-tests

Overall attitudinal differences were first tested by multivariate analysis of variance with 15 items serving as dependent variables and resident research status as an independent variable. This analysis revealed significant omnibus effect of the resident research status across all 15 items [F(15, 36) = 2.96, p = .005], suggesting that the responses of the two groups of residents were significantly different. The univariate follow-up tests identified 6 statements for which the mean scores of the research stream residents differed significantly from those of other residents at p < .01. Not surprisingly, the research stream residents had significantly higher mean scores on the item "I would like to get more research training as part of the family medicine research training" and significantly lower scores on the item "I am not interested in research activities" than those residents not in the research stream. They were also significantly less likely to agree with the statement that "Research is adequately emphasized in family medicine training at McMaster". The responses of the research stream residents also differed significantly, and in the expected direction, with regard to 3 general statements concerning primary care research. While not significantly different at p < .01, the research stream residents were less likely to agree with the statement that family medicine faculty at McMaster are knowledgeable about research and that there is sufficient statistical/epidemiological support available to residents who are interested in research. They were also somewhat more likely to agree that required research projects should be part of residency training.

Biostatistical and epidemiological knowledge

Section 3 involved 18 MCQ questions testing knowledge of research methodology, epidemiology and biostatistics and it's application to the solution of problems. The mean percent MCQ knowledge score was 66.0% (SD = 17.8%). Questions regarding sampling strategies, statistical tests such as chi square, epidemiological definitions such as sensitivity, specificity, predictive value, validity, attributable risk, odds ratio, mean, mode, median, standard deviation were answered correctly by at least 60% of residents. Poorly answered questions included items on research ethics, research terminology such as triangulation, t-tests, cost-benefit analysis, incidence vs. prevalence vs. mortality/case fatality rates. The overall mean score of the research stream residents while more than 10 percent higher than those of other residents did not reach statistical significance (mean 74.7%, SD= 10.4% vs. mean 63.7%, SD = 18.8%; t = 1.9, df = 50, p = .07). Questions answered correctly by residents not in the research stream were answered correctly by slightly higher percentage of residents in the research stream while poorly answered questions followed a similar pattern in both groups.

Discussion

The internal/external validity of this study may be limited by several factors. First, this was a self-report survey. Second, the survey instrument itself was not completely validated. Third, because the maximum number of respondents was predetermined, some important differences in terms of background characteristics, research attitudes, or self-reported knowledge might have be missed because of inadequate statistical power. Finally, the sample surveyed might not be representative of family medicine residents elsewhere in North America.

Despite the above limitations certain findings can be restated: the vast majority of the residents reported previous research experience and/or some training in epidemiology and biostatistics and there were no differences in that regard between those in the research stream and those in the regular curriculum. Although most residents had positive attitudes towards primary care research, those who self-selected themselves to be part of the research stream tended to be more positive. Statistically significant differences were found between those in the research stream and those in the regular program in terms of research interest, interest in obtaining more research training while a resident, importance of developing research early in medical education, and adequacy of research emphasis at McMaster. In other words, this study suggests that it comes down to interest in research, rather than knowledge, skills, or prior research involvement. While such conclusions are perhaps self-evident to some, they underscore the larger problem of family medicine research in North America.

Residents in the research stream had stronger views regarding perceived lack of support staff and lack of time for research. Temte et al also found that residents who were actively involved in a research project identified the importance of time and resource personnel to assist with research design and data analysis more than residents who were interested in pursuing research during residency, but were not currently active [3]. It was also surprising that those in the research stream were no more confident in their research skills than those in the regular stream even though they tended to perform better on the knowledge questions.

The study by Temte et al, in which they examined factors associated with research interest and activity in Wisconsin family medicine programs, reported that 85.3% residents felt that research experience was desirable for family physicians [3]. However, many McMaster residents remained neutral or disagreed with the statement that family medicine residents should complete research projects during their training or should be assigned a research mentor. Even those in the research stream were relatively lukewarm in their views on these topics (mean = 6.2, SD = 3.1 and mean = 6.0, SD = 2.9; respectively). As in the Wisconsin study, those residents who were interested in research tended to be of lower training year and female [3].

Studies indicate that residents differ in their attitudes towards required research projects. In a study by Konen et al, half of the family medicine residents from a community hospital based, university affiliated training program in Syracuse, NY felt that the exercise was an unnecessary annoyance [18]. A survey of family practice program's graduates who completed their training between 1987–1992 at the University of Toronto and who had participated in a required academic project during their residency program indicated that most research projects had been literature reviews. Furthermore, the majority of respondents felt neutral or negative about their projects and 79% would not have done a project if it had been optional [19]. This differs from a 1993 study that described a six-year experience with required research projects in a pediatric residency program. A questionnaire administered to residents after the completion of their project and presentation, found that 62% believed completing a research project was quite worthwhile, and only 19% felt unfavorable about the project [9].

Research training during residency can provide a structured environment where research activity is encouraged and valued and necessary resources are likely to be available. Most family medicine residency programs in North America have addressed the issue of strengthening the research base of primary care research by implementing required research projects, generally with mixed results. Our results suggest that there is a demand for enhanced research experience for a selected sub-group of family medicine residents. The research stream approach implemented at McMaster University might offer unique opportunities not only to optimize the use of limited research resources but also to create a model of multidisciplinary collaborative primary care research training program with potential applications in other residency programs. This baseline assessment was important to our understanding of the similarities and differences in those residents who self-selected to join our research stream at the inception.

Pre-publication history

The pre-publication history for this paper can be accessed here: