Variability in research productivity among Canadian surgical specialties.

Background: Academic productivity, as measured by number and impact of publications, is central to the career advancement and promotion of academic surgeons. We compared research productivity metrics among specialties and sought factors associated with increased productivity.
Methods: Academic surgeons were identified through departmental webpages and their scholarly metrics were collected through Scopus in a standardized fashion. We collected total number of documents, h-index, and average number of publications per year in the preceding 5 years. We explored whether presence of a training program, graduate degree, academic rank and size of the clinical group affected productivity metrics. Linear regression was used for multivariable analysis.
Results: We collected data on 2172 surgeons from 15 separate academic centres across Canada. Wide variability existed in metrics among specialties, with cardiac and neurosurgery being the most productive, and vascular surgery and plastic surgery being the least productive. The average number of publications was 71, and the average h-index was 18.7. The average h-index for cardiac surgery was 25.7 compared with 8.3 for vascular surgery (p < 0.001). Our multivariable model identified academic rank, surgical specialty, graduate degree, presence of a training program, and larger clinical group as being associated with increased academic productivity.
Conclusion: There is variability in research productivity among Canadian surgical specialties. Cardiac surgery and neurosurgery are productive, whereas vascular surgery and plastic surgery are less productive than other surgical disciplines. Obtaining a research-oriented graduate degree, being part of a larger clinical group, and presence of a training program were all associated with higher productivity, even after adjusting for academic rank and specialty.

Academic surgeons must fulfill numerous professional obligations. In addition to the care of complex surgical patients, they are expected to mentor and train surgical residents and conduct research. Research output, as measured by number and impact of publications, is central to the career advancement and promotion of academic surgeons.1–3 Simply relying on a surgeon’s total number of publications does not convey a true representation of their contribution to the academic literature as practice-changing randomized controlled trials will be counted the same as case reports. Along with the number of publications, h-index has emerged as a reliable metric to measure the average impact of a given surgeon’s publications.4 Briefly, the h-index is a measure of an author’s personal impact and is defined as “x number of publications cited at least x number of times.” It has been used extensively in the academic productivity literature to help evaluate the impact of a given surgeon’s research performance.5 Although specialty-specific studies looking at academic performance have been conducted in Canada,6 to our knowledge, no study has conducted a national comparison of different surgical specialties with regards to average academic productivity metrics. We hypothesized that there would be large discrepancies in average research productivity among surgical specialties in Canada. We conducted a nation-wide study to compare average research productivity among surgical disciplines and identify predictors of increased academic productivity. This is important as specialty-specific benchmarks should be used when evaluating surgeons for academic job retention and promotion. In addition, less productive specialties may look to more productive specialties for ways to improve their research output.

Methods

We collected data on academic surgeons from the specialties of cardiac surgery, thoracic surgery, neurosurgery, plastic surgery, vascular surgery, general surgery, orthopedic surgery, and pediatric surgery practising at 15 university-affiliated medical centres across Canada (Appendix 1, available at canjsurg.ca/016319-a1). Through institutional and departmental websites, individual surgeons’ data, including name, gender, specialty, academic rank, graduate degrees held, and the presence of an onsite resident training program were gathered. We excluded surgeons with adjunct appointments. This study protocol was reviewed by Western University’s ethics review board and deemed not to require formal board review as the data were publicly available and we planned to present the data as deidentified summary data only.

Academic productivity metrics, including total publications, h-index, and yearly average number of publications in the preceding 5 years, were obtained from the Scopus database using the author search function, by surgeon first and last name as well as affiliated university (last updated September 2018). Although alterative search strategies exist, Scopus has been used widely in the scholarly output literature and has been shown to correlate well with Google Scholar searches.7 In instances where the search returned multiple results with the same first and last name, a case by case decision was made based on comparing each author’s affiliations, publications and coauthors to the specialty and declared research interests. It is important to note that we measured only number of publications and their potential impact (h-index); we did not account for other metrics of academic activity, such as teaching, committee work, or leadership, and thus only partially captured a given surgeon’s academic contributions.

The surgeons were grouped according to various factors, including specialty, institution, academic rank, graduate degrees held, and the presence of an onsite resident training program. To determine whether there were significant differences in the outcomes (total publications, h-index, average yearly publications in preceding 5 years) between different groups, we conducted 1-way analysis of variance (ANOVA) with post hoc Tukey tests and t tests, as appropriate. To assess which surgeon- or practice-related factors were independently associated with productivity, linear regression was used with the dependent variables log-transformed given the non-normal distributions of the outcome variables. All analyses were 2-sided with significance set at p < 0.05 and were performed using SPSS version 24 (IBM).

Results

We collected data from 15 academic centres encompassing 114 individual surgical departments for a total of 2172 surgeons (1825 men [84%] and 347 women [16%]). The largest specialties represented were orthopedic surgery (n = 631) and general surgery (n = 590), while the smallest specialties were vascular surgery (n = 92), thoracic surgery (n = 74) and pediatric surgery (n = 31) (Table 1). Of the surgeons whose academic rank we were able to obtain, 49.5% (n = 821) were assistant professors, 25.1% (n = 416) were associate professors, 20.2% (n = 335) were full professors, and the remaining 5.2% were lecturers (n = 43) and emeritus professors (n = 43).

Table 1

Research productivity metrics by surgical specialty in Canada (arranged by average total number of documents from highest to lowest)

Specialty	Total number of publications, mean (SD)	H-index, mean (SD)	Average number of publications/year, mean (SD)	Number of publications/5 years, mean (SD)
Neurosurgery (n = 189)	72.4 (126.1)	16.9 (23.4)	4.1 (7.6)	20.7 (38.2)
Cardiac surgery (n = 145)	71.4 (85.8)	18.7 (16.1)	3.4 (4.8)	17.2 (24.0)
Urology (n = 218)	60.7 (88.0)	14.2 (15.1)	3.6 (5.4)	18.3 (27.2)
Pediatric surgery (n = 31)	55.4 (85.1)	12.9 (13.1)	3.3 (4.4)	16.7 (21.9)
Thoracic surgery (n = 74)	52.1 (77.7)	13.6 (14.4)	3.3 (5.3)	16.4 (26.7)
Orthopedic surgery (n = 631)	31.3 (59.8)	9.3 (11.8)	2.0 (3.8)	10.1 (19.1)
General surgery (n = 590)	28.3 (55.8)	8.8 (12.2)	1.6 (3.0)	7.7 (15.2)
Vascular surgery (n = 92)	25.7 (40.2)	8.3 (8)	1.4 (2.4)	7.1 (11.9)
Plastic surgery (n = 202)	24.1 (43.5)	7.7 (9.2)	1.4 (2.5)	6.8 (12.5)

SD = standard deviation.

For all 2172 surgeons, the average h-index was 11.0, average total publications was 40.1, and average publications per year in the preceding 5 years was 2.45. However, all 3 of these metrics were heavily left-skewed, and the mode for h-index, total publications and publications per year in the preceding 5 years were all equal to 0 (Figure 1). This indicates that overall there are a few highly productive surgeons and that a larger number of surgeons who hold academic appointments publish very little or not at all. This pattern was consistent throughout the specialties.

Fig. 1

Frequency distribution of academic metrics for the entire cohort of surgeons.

All 3 measures of academic productivity differed significantly among specialties (p < 0.001) (Table 1). When we examined specific comparisons using post hoc Tukey tests, significant differences existed when comparing high-publishing specialties (neurosurgery, cardiac surgery, urology) with low-publishing specialties (general surgery, orthopedic surgery, plastic surgery and vascular surgery). On average, neurosurgeons had 72.4 publications and cardiac surgeons had 71.4 publications, whereas plastic surgeons had 24.1 publications and vascular surgeons had 25.7 publications (Table 1). The differences in h-index were similar, with neurosurgeons (16.9) and cardiac surgeons (18.7) having significantly higher h-indices than with vascular surgeons (8.3) and plastic surgeons (7.7).

When we explored differences in number of publications per academic rank, we saw a similar pattern, with low-publishing specialties having fewer publications and a lower h-index per academic rank than high-publishing specialties. For example, neurosurgeons at the rank of associate professor had on average 51.5 publications with an h-index of 14.5, while vascular surgeons at the same rank had only 27.5 publications with an h-index of 9.8. Similarly, neurosurgeons at the rank of full professor had on average 189.4 publications with an h-index of 34.7, while vascular surgeons at the same rank had only 68.5 publications and an h-index of 16.8 (Table 2).

Table 2

Academic metrics by rank and specialty

Specialty	Assistant professor			Associate professor			Professor
	Total	H-index	Avg/yr	Total	H-index	Avg/yr	Total	H-index	Avg/yr
Neurosurgery	30.8	9.9	2.3	51.5	14.5	3.6	189.4	34.7	9.3
Cardiac surgery	21.1	7.6	1.7	56.2	15.7	3.9	98.3	32.1	5.1
Urology	19.5	5.6	1.6	56.8	15.5	4.3	143.3	29.2	6.9
Pediatric surgery	26.5	6.9	2.5	41.2	12.8	2.9	249.7	41	8.7
Thoracic surgery	14.4	5.8	1.3	46.3	13.9	3.5	136.0	28.5	7.7
Orthopedic surgery	14.5	5.1	1.5	41.2	12.4	3.3	138.2	31.1	6.9
General surgery	9.7	4.3	0.8	23.2	8.2	1.6	117	28.0	4.7
Vascular surgery	13.1	5.5	1.1	27.5	9.8	1.2	68.5	16.8	2.7
Plastic surgery	8.7	3.8	0.7	24.9	9.0	1.5	66	17.6	3.3

In examining academic productivity among different academic ranks for the entire group, we found a statistically significant difference (p < 0.001) in h-index, total publications, and average yearly publications in the preceding 5 years among full professors, associate professors and assistant professors (Figure 2). For the purposes of this comparison, lecturers and emeritus professors were excluded and department chairs were included as full professors. When testing for differences in academic productivity based on graduate degrees, all 3 metrics showed statistically significant differences (p < 0.05) among surgeons with no graduate degree, surgeons with an MSc and surgeons with a PhD. Other graduate degrees, including MEd, MHPE, MBA, and MPH, showed no significant difference when compared with other groups (Figure 3). When we compared academic productivity at centres with an on-site resident training program to centres without one, t tests showed statistically significant differences (p < 0.001) in all 3 metrics (Figure 4).

Fig. 2

Comparison of research output metrics by academic rank for the entire cohort of surgeons.

Fig. 3

Comparison of h-index by graduate degree obtained.

Fig. 4

Effect of the presence of a training program on academic metrics.

Linear regression analysis using 3 separate models for the 3 outcomes in question found specialty, academic rank, presence of graduate degree, larger division size, and presence of on-site training program to be independently associated with increased academic output for all 3 metrics of research productivity, whereas gender and academic institution were not associated with increased output (Appendix 1, Table S1).

Discussion

We identified a large discrepancy in average research productivity among surgical specialties across Canada. Cardiac surgery, neurosurgery and urology were all relatively more productive according to all 3 metrics (total number of publications, h-index, and average number of yearly publications in the preceding 5 years) when compared with other specialties; general surgery, orthopedic surgery, plastic surgery and vascular surgery were the least productive. These differences were significant even after adjusting for academic rank, institution, presence of graduate degree, and presence of on-site training program. Reasons for this discrepancy are not fully explained by our data. When looking at specific factors associated with increased research productivity, surgeons with an MSc or PhD were more productive, whereas those with an MEd, MPH or MBA were not significantly more productive than those without a masters degree. In an analysis of research productivity of residents from 2 academic centres in Canada, Merani and colleagues found residents who completed a graduate degree during training were more productive academically both during and after training, particularly when they completed a PhD.8 When looking at differences in the proportion of surgeons with research-oriented graduate degrees (PhD or MSc), we found that most neurosurgeons (55.7%) have graduate degrees, which is higher than some of the less productive specialties such as general surgery (32.1%) and vascular surgery (37%). This may help to explain why neurosurgeons were more productive; however, this is not the sole reason for interdisciplinary differences in research output, as urologists had the lowest proportion of graduate degrees (21.8%) but were among the higher academically productive specialties (Appendix 1, Table S2).

Other authors have noted differences in average research productivity when comparing different surgical specialties. Ashfaq and colleagues, in an analysis of more than 3000 US surgeons, found significant differences in h-index between specialties.5 They identified surgical oncology and thoracic surgery to have higher h-indices than other specialties, particularly colon and rectal surgery and trauma surgery. Similarly, Valsangkar, in an analysis of 3850 academic surgeons from the top 15 academic institutions in the US, found cardiothoracic surgery, surgical oncology and transplantation to have higher research output when compared with other specialties.1 Interestingly, Ashfaq and colleagues found that vascular surgeons were among the more highly productive specialties, with a mean h-index of 16.7, whereas in our study they were near the bottom, with a mean h-index of 8.3. This suggests that the average American academic vascular surgeon is nearly twice as productive/impactful as the average Canadian academic vascular surgeon. Why these cross-border differences exist is unclear, although indirect evidence from volume–outcome studies suggests that the average Canadian vascular surgeon is clinically busier than their American counterparts, as evidenced by higher per-surgeon abdominal aortic aneurysm repair volumes.9 While these cross-border differences in academic output by specialty exist for vascular surgery, this may not apply to all specialties. In a separate analysis of 2429 academic surgeons from 20 randomly selected academic institutions in the US, the average h-index for a plastic surgeon was 8.4, which is very similar our finding of 7.7.10 This indicates that in the US, like in Canada, plastic surgeons publish fewer papers than surgeons from other specialties.

Both regional and specialty-specific factors need to be taken into account when assessing a given surgeon’s relative academic output compared with that of his or her peers. Some authors have argued that the h-index may be influenced by the size of the specialty, with more of a target audience for reading and citation of one’s work, thus increasing the h-index. In an analysis of US data, general surgeons were found to have significantly higher h-indices than surgeons from smaller specialties.10 In our data both orthopedic surgery and general surgery were relatively underproductive compared with the smaller specialties of neurosurgery and cardiac surgery, indicating that in Canada, size of specialty is not a major driver of academic output or impact.

We found presence of an on-site training program to be associated with increased productivity. Presumably, the access to residents and trainees allows surgeons to mentor them through research projects and increase their output. Although this was significant in the analysis, when examined among specialties no consistent pattern emerged, as both orthopedic surgery and general surgery programs had on-site training programs but were relatively underproductive. The culture and research expectations may differ among training programs, and this may explain why they do not track perfectly with specialty output. One Canadian training program was able to increase their research output by adopting a formal resident research program consisting of formal meeting and mentoring schedules, research-related didactic teaching, and clear goals and deliverables.11 In a separate study, otolaryngologists who mentored medical students were more productive academically than those who did not.12

Being part of a larger clinical group was associated with increased productivity. We identified a consistent pattern of research productivity among the various specialties with each having a few extremely productive members (publications in the 200–400 range) and a larger number of minimally productive surgeons. Presumably, being in a larger clinical group allows a research-focused surgeon to better allocate their time toward producing more research output while they are protected from clinical obligations. Many academic departments stress the importance of “protected time” for academic work, facilitating weekly work schedules that allow surgeons to devote time to research. Interestingly there is very little evidence to support this practice. In fact, a study of cardiac surgeons from the state of New York found no overall correlation between number of publications and clinical volume. They identified a sub-group of surgeons (15%) who had a high number of publications and who were also the busiest clinically, with the exception of a few PhD surgeons (1%) who were clearly full-time researchers.13 More research is needed on the potential relationship between clinical volume/workload and academic productivity. Many academic departments will use clinical earnings to support surgeons with a research focus, expecting increased academic output, but it is unclear whether this strategy consistently leads to more impactful research. Given that most academic surgeons still depend on their clinical work to generate revenue, some departments have used academic point systems to help quantify and remunerate surgeons for academic work, which has led to increased productivity.14 We did not assess the impact of alternate funding strategies on academic productivity, as individual surgeon salary data are unavailable.

It is not surprising that academic rank is associated with increased research output, as these are precisely the metrics that factor heavily in a surgeon’s promotional application.10,15 What is interesting is the significant discrepancy in average research metrics between the specialties. For a cardiac surgeon promoted to the rank of associate or full professor, they have on average 56 total publications and 133 total publications, respectively, while for vascular surgery the average is 27 publications for associate professors and 68 publications for full professors. This highlights the need to consider specialty-specific benchmarks when considering surgeons for academic promotion. We have tabulated average metrics to help guide academic promotion committees when assessing a surgeon’s research productivity and comparing it to that of their peers (Table 2). Why this large discrepancy exists is not entirely clear from our data. Some of it may be attributed to the slightly increased prevalence of graduate degrees and training programs among specialties, but other factors are also important. Perhaps the less productive specialties are busier clinically; however, as discussed earlier the link between protected time and academic productivity has yet to be conclusively shown. Other factors that likely contribute are the availability of mentors, research networks, and access to basic science partners, which all may differ among specialties. Other factors such as institutional and specialty culture likely play a role as well. Future work using tools like network analysis may shed some light on other factors that can explain these differences.

Despite the differences in absolute numbers of publications and impact among specialties, the distribution of academic productivity was similar among specialties, with each containing a few very productive surgeons and a larger number of academic surgeons with very few publications. This skewed distribution of academic activity was also evident in a study of 3850 academic surgeons from the US, where the top 10 most cited faculty within each department contributed 42% of all publications and 55% of all citations.1 Implications of these findings include leveraging these key individuals to lead departmental research endeavours and mentor younger surgeons to better prepare them for academically productive careers.

Limitations

Our study has some limitations. We did not capture information on the number of first/last author publications. It is possible that a subset of surgeons with a large number of publications are simply benefiting from membership in active research groups but are not driving the research themselves; however, this would likely apply to all specialties. Also the h-index is vulnerable to self-citation, which can artificially increase the impact of an author’s work.16 We also were unable to explore individual surgeons’ clinical workload, and more work needs to be done to explore the potential relationship between clinical activity and academic productivity. We also did not quantify other academic obligations such as committee work, volunteer work, and teaching, all of which make up important academic contributions. It is probable that less research-focused surgeons are contributing in other ways to the academic mission of their respective institutions, and this was not captured in our study; however, this would have applied to all specialties equally. We also did not capture information on grant funding; however, this is often correlated with the publication metrics used here.1 We relied on Scopus and departmental websites for our data gathering, and unfortunately these can sometimes be outdated,6 which may have led to some underestimation of research output or misclassification of rank or graduate degree; however, this would have applied similarly to all specialties.

Conclusion

There is wide variability in average research productivity among Canadian academic surgeons. Cardiac surgery, neurosurgery and urology were relatively productive specialties, whereas vascular surgery and plastic surgery published fewer papers on average than other surgical disciplines. Specialty-specific benchmarks should be considered when evaluating surgeons for academic promotion. Obtaining a research-oriented graduate degree, being part of a larger clinical group, and presence of an on-site training program were all associated with higher productivity, even after adjusting for academic rank, institution, and specialty.