The KRESCENT Program (2005-2015): An Evaluation of the State of Kidney Research Training in Canada.

BACKGROUND: The Kidney Research Scientist Core Education and National Training (KRESCENT) Program was launched in 2005 to enhance kidney research capacity in Canada and foster knowledge translation across the 4 themes of health research.
OBJECTIVE: To evaluate the impact of KRESCENT on its major objectives and on the careers of trainees after its first 10 years.
METHODS: An online survey of trainees (n = 53) who had completed or were enrolled in KRESCENT was conducted in 2015. Information was also obtained from curriculum vitae (CVs). A bibliometric analysis assessed scientific productivity, collaboration, and impact in comparison with unsuccessful applicants to KRESCENT over the same period. The analysis included a comparison of Canadian with international kidney research metrics from 2000 to 2014.
RESULTS: Thirty-nine KRESCENT trainees completed the survey (74%), and 44 trainees (83%) submitted CVs. KRESCENT trainees had a high success rate at obtaining grant funding from the Canadian Institutes of Health Research (CIHR; 79%), and 76% of Post-Doctoral Fellows received academic appointments at the Assistant Professor level within 8 months of completing training. The majority of trainees reported that KRESCENT had contributed significantly to their success in securing CIHR funding (90%), and to the creation of knowledge (93%) and development of new methodologies (50%). Bibliometric analysis revealed a small but steady decline in total international kidney research output from 2000 to 2014, as a percentage of all health research, although overall impact of kidney research in Canada increased from 2000-2005 to 2009-2014 compared with other countries. KRESCENT trainees demonstrated increased productivity, multiauthored papers, impact, and international collaborations after their training, compared with nonfunded applicants.
CONCLUSIONS: The KRESCENT Program has fostered kidney research career development and contributed to increased capacity, productivity, and collaboration. To further enhance knowledge creation and translation in kidney research in Canada, programs such as KRESCENT should be sustained via long-term funding partnerships.

What was known before

The Kidney Research Scientist Core Education and National Training (KRESCENT) Program was launched in 2005 to increase kidney research capacity in Canada and enhance collaborations and knowledge translation. A preliminary review of the program in 2010 revealed enhanced training positions and high success rates for trainees in obtaining grants and academic appointments. The 10-year anniversary in 2015 provided an opportunity for a formal evaluation of KRESCENT, using qualitative and quantitative data.

What this adds

Survey data and a bibliometric analysis reveal that KRESCENT has led to increased numbers and quality of research publications, and promoted the development of a collaborative community of leading kidney research investigators. KRESCENT has significant potential to improve the lives of people affected by kidney disease in Canada and should be sustained via long-term funding partnerships.

Background

The Kidney Research Scientist Core Education and National Training (KRESCENT) Program was established in Canada in 2004, with enrollment of the first trainees in July 2005. KRESCENT originated with recognition by the Canadian research community of the rising prevalence of kidney disease, coupled with declining interest and engagement of trainees in kidney research.[1,2] At its outset, the major objectives of the program were to enhance kidney research capacity in Canada by training the next generation of leading investigators and to improve collaborations and knowledge translation (KT) across the 4 themes of health research: biomedical, clinical, health systems and services, and social, cultural, and environmental factors that affect the health of populations.

Since 2004, KRESCENT has been sustained by a unique partnership funding model that includes the Kidney Foundation of Canada (KFOC), the Canadian Society of Nephrology (CSN), and the Canadian Institutes of Health Research (CIHR) Institute of Nutrition, Metabolism, and Diabetes. Salary support is provided for up to 3 years to Post-Doctoral Fellows (MD and/or PhD), New Investigators (MD and/or PhD, within the first 3 years of academic appointment at a Canadian institution), and Allied Health Scholars (enrolled in graduate PhD programs in Canada). Since 2005, the funding model has been supported by contributions (>$3.5 M) from the private sector and other granting agencies (Additional file 1). Seventy-two training positions have been funded in KRESCENT since 2005 with representation in 7 provinces. Fifteen Post-Doctoral Fellows have received KRESCENT salary support for training at academic centers outside Canada. Allied Health Scholars have been relatively underrepresented within the program (6 trainees since 2005), although changes have been incorporated into the peer review selection process in recent years to increase participation by this group of health professionals.

A unique feature of KRESCENT is its core curriculum, which is delivered at 2 workshops annually. The curriculum focuses yearly on selected major topics in kidney research and consists of core lectures from content experts spanning biomedical science to topics in health service delivery and population health, group discussions around pertinent journal articles, and oral presentations by trainees. Career development sessions and grant- and manuscript-writing exercises are also built into the curriculum. KRESCENT also provides mentorship support to trainees, particularly at the New Investigator level, via biannual meetings with senior investigators involved in the program, and review of a checklist of topics relevant to career development.

A preliminary review of the program in 2010 found that numbers of training positions in kidney research had increased due to KRESCENT and that trainees had demonstrated a high success rate in obtaining peer review grant support and academic appointments.[3] Now that 10 years have passed, we conducted a formal evaluation of KRESCENT to determine whether the program is meeting its objectives. The evaluation included both qualitative and quantitative components, using a survey and bibliometric analysis. Furthermore, we assessed the overall state of kidney research in Canada since 2000, in comparison with other countries, to frame the impact of KRESCENT in this context.

Methods

All recipients of KRESCENT awards between 2005 and 2014 inclusive were contacted by e-mail in the summer of 2015 and asked to complete an online survey questionnaire. The survey consisted of 14 questions that covered a variety of areas, including success rate at obtaining CIHR or other peer review operating grants, and numbers of publications or peer review grants with other KRESCENT trainees (see Additional file 2). Electronic reminders were sent out to those who did not complete the survey, and the survey was closed after 6 weeks. Survey results are reported either as categorical values, or in some cases, narrative responses were grouped into common themes identified by the authors, and arbitrarily selected responses depicted. In addition to the survey, KRESCENT trainees were asked to submit their updated curriculum vitae (CVs) to KFOC for review. The CVs were reviewed, and data were collected regarding demographics, research theme and area of research, faculty appointment status, and grants/publications. All trainees consented to the use of anonymized submitted data and survey responses for publication for this formal evaluation.

A bibliometric analysis of KRESCENT trainees, before and after their training, in comparison with unsuccessful applicants to the KRESCENT Program in the same period was also conducted. This analysis compared, for the 2 groups, the average number of papers, average number of authors per paper, international collaborations (as determined by presence of international addresses in addition to Canadian addresses on the manuscript), average of relative citations (ARC), and average of relative impact factor (ARIF). ARC measures the frequency in which the article is cited by other researchers, whereas ARIF indicates the overall prestige of the journal in which the manuscript was published.[4] Both indicators control for the disciplinary differences in papers’ citation rates. Only articles and reviews were included in the analysis: Abstracts, posters, presentations, and conference proceedings were excluded.

To contextualize the results, data were compiled on the field of kidney research at the world level, providing insight into the impact of all Canadian kidney researchers (includes KRESCENT trainees and other kidney researchers) compared with kidney researchers from the top 20 countries. Papers considered as belonging to the area of kidney research were those that had either one of the specific words in their title or keywords attributed, or that were published in one of the specialized journals (Additional file 3). Two time periods were examined (2000-2005 and 2009-2014) to assess impact before and after KRESCENT. Finally, the overall impact of kidney research relative to research in other health sectors was assessed over time.

Results

Data From Online Survey and CV Review

From 2005 to 2014 inclusive, there were 66 KRESCENT training awards allocated (37 Post-Doctoral Fellowships, 23 New Investigators, and 6 Allied Health Scholarships) to 53 individual recipients (some individuals received both Post-Doctoral Fellowships and subsequent New Investigator Awards). Thirty-eight awards were allocated to researchers with MD degrees, whereas 22 awards went to non-MD, PhD scientists. Of the awards to MD researchers, 30 (78.9%) were allocated to adult nephrologists, whereas 8 (21.1%) were granted to pediatric nephrologists. The 6 Allied Health Scholarships were held by awardees in Clinical Psychology (2), Health Services (2), Rehabilitation Sciences (1), and Systems Design Engineering (1).

In all, 45 individuals responded to at least 1 question on the online survey (85%), and 39 trainees completed the entire survey (74%). The 14 awardees who did not respond to the survey represented a mix of individuals in each of the 3 program categories (9 Post-Doctoral Fellows, 3 New Investigators, and 2 Allied Health Scholars).

Forty-four (83%) trainees submitted CVs. Of the respondents, 79% had secured operating grant funding from CIHR as a principal investigator (PI) or co-investigator since completion of their training (41% had funding from CIHR as a PI or co-PI). Ninety percent of respondents reported that the KRESCENT Program had contributed importantly to their ability to obtain CIHR funding, whereas 75% noted that KRESCENT contributed to some extent or to a great extent in their success at obtaining other peer review grant support.

Three-quarters (76%) of Post-Doctoral Fellows reported that they had been appointed to academic positions at the Assistant Professor level, within an average of 7.8 months of completing training.

The areas in which KRESCENT participants were working are shown in Figure 1. The leading 5 diseases/conditions reported to be impacted by KRESCENT research were chronic kidney disease (CKD), kidney biology, diabetes, acute kidney injury, and transplantation.

Figure 1.

Research areas impacted by KRESCENT. Bar graph depicts responses to the survey question “Please select the disease or research area impacted by your research.” Percentage of responses is on y-axis (n = 44 responses), and number of responses is indicated in parentheses above each column. “Other” category included renal failure (4), self-reporting of obesity/nutrition (1), ethics (1), drug-induced nephrotoxicity (1), urology (1), patient engagement (1), medication adherence (1), and development of artificial kidneys (1).

Note. KRESCENT = Kidney Research Scientist Core Education and National Training.

More than 9 in 10 (93%) respondents noted that KRESCENT training had contributed to the achievement of research outcomes, with the majority reporting “knowledge creation” (93% of respondents) and “development of new research methods” (50% of respondents) as the leading outcomes (40 trainees responded to this survey question, and thus, the calculation of proportions is based on a denominator of 40, instead of 39).

The development of collaborative research teams to enhance KT is one of the goals of KRESCENT. The CIHR research cycle categorizes KT1 as the opportunity to define research questions and methodologies. KT2 involves conducting research (participatory role), and KT3 refers to publishing research findings in plain language and accessible formats.[5] In response to the question “To what extent are you involved in KT and specifically KT1-, KT2-, or KT3-level translational research?” the answer was “to some extent” or “to a great extent” for 45% of respondents for KT1, 43% for KT2, and 30% for KT3. More than 4 in 5 (82%) respondents reported that since completion of training, they conducted research as part of a multidisciplinary team. Interestingly, respondents reported an average of 3.4 publications per trainee (range, 0-36) and an average of 1.2 peer review grants per trainee (range, 0-17) arising from collaboration with other KRESCENT awardees.

There were 39 responses to the question “What aspect of the KRESCENT Program did you value the most?” (Table 1). Three major themes emerged from these responses: (1) the opportunity for scientific interaction and networking (29 responses); (2) the value of the workshop meetings, including grant-writing exercises (14 responses); and (3) mentorship support (10 responses).

Table 1.

Sample Responses to “What Aspect of the KRESCENT Program Did You Value the Most?”

Theme	Sample responses
Scientific interaction/networking (29/39 responses)	- “Foster close relationships and collaboration with the KRESCENT community”- “Networking opportunities to meet researchers from other disciplinary fields”- “Networking with other kidney researchers across Canada”- “The network I have built will last a lifetime”- “The interaction with others in other pillars of research”
Workshops/exercises (14/39 responses)	- “Interdisciplinary research training”- “Grant and publication review”- “Regular face-face meetings with a focus on renal-specific topics”- “All aspects of the training workshops were immensely helpful”
Mentorship (10/39 responses)	- “Les conseils de nos mentors”- “Amazing mentors”- “Opportunity for mentorship outside of your direct research supervisor”- “Formalized mentorship”

Note. KRESCENT = Kidney Research Scientist Core Education and National Training.

There were also 39 responses to the question “What aspect of the KRESCENT Program did you value the least?” (Table 2). The predominant theme that emerged related to assignments and workload associated with the curriculum (17 responses). Ten of 39 respondents reported that there were no negative aspects.

Table 2.

Sample Responses to “What Aspect of the KRESCENT Program Did You Value the Least?”

Theme	Sample responses
Workshop assignments (17/39 responses)	- KIMs and KAMs[a] - “Some of the required activities (homework)”- “Written assignments”- “The homework, which was not useful to me”- “Some of the take-home assignments”
No issues to report (10/39 responses)	- “None”- “None—It was all valuable”- “I think I enjoyed everything, sincerely!”- “All had some value”
Miscellaneous (9/39 responses)	- “Some of the didactic sessions on very specific topics were not that useful”- “Traveling”- “Having each entire year dedicated to only one aspect of kidney disease”- “Not much training related to technology development”

KIMs and KAMs refer to Knowledge Integration Modules and Knowledge Application Modules, respectively, which were written take-home assignments in the early years of KRESCENT.

Note. KRESCENT = Kidney Research Scientist Core Education and National Training.

The final question of the survey asked for any additional comments, and 25 individuals responded. In general, the comments provided were highly positive. For example, one trainee noted, “KRESCENT had a huge impact on my career development. Having the award gave me a level of credibility and visibility in the Canadian kidney research community that would have been difficult to achieve in any other way. There were numerous intangible benefits.”

Bibliometric Analysis

The bibliometric analysis revealed that international kidney-related research productivity has experienced a small but steady decline relative to other health sectors from 2000 to 2014 (4.5%-3.9%), as determined by numbers of published manuscripts (Figure 2). In 2000, Canada ranked eighth in the world in terms of number of kidney research publications (n = 600), and in 2014, Canada had almost doubled the number of kidney research publications (n = 1124), ranking seventh in the world behind the United States, China, Japan, Germany, the United Kingdom, and Italy.

Figure 2.

Decline in total international kidney research manuscripts from 2000 to 2014. Graph depicts gradual decrease in the percentage of kidney research publications from 2000 to 2014, as the percentage of all health research publications. Below graph is table illustrating numbers of publications per year and the percentage of yearly publications devoted to kidney research. Data derived from top 20 countries involved in health research. Kidney research manuscripts decreased from 4.5% to 3.9% of total health research manuscripts from 2000 to 2014.

In comparison with other major countries, the relative impact of Canadian kidney research increased in the period of 2009-2014, compared with 2000-2005. Figure 3 illustrates a positional analysis of the top 20 countries publishing in kidney research as a function of the mean impact of their papers and the level of specialization in the field of kidney research. The ARC for Canada between 2000 and 2005 was 1.32, and in 2009-2014, it increased to 1.90. ARC measures the frequency in which the article is cited by other researchers, whereas the Specialization Index (SI) is a measure of the degree of specialization in a scientific discipline of a given entity in terms of world science. When the SI is greater than 1, it indicates that the country of interest is more specialized in its priority area, compared with the world average. Similarly, when the ARC is greater than 1, the papers are cited more frequently than the world average in its priority area. For the period between 2009 and 2014, note that the ARC and SI include data from all kidney researchers in Canada, not just KRESCENT awardees.

Figure 3.

Positional analysis of kidney research in 20 countries from 2000 to 2005 (A) and 2009 to 2014 (B). The y-axis shows ARCs for manuscripts, as an index of impact. The x-axis depicts the SI: When the SI is greater than 1.0, the country produces more kidney research manuscripts than expected, and the opposite is true when the index is less than 1.0. For Canada, both ARC and SI for kidney research (ie, includes KRESCENT trainees and all other kidney investigators) increased in 2009 to 2014, compared with 2000-2005.

The findings for a bibliometric analysis conducted for KRESCENT awardees before and after their training, and for applicants to KRESCENT who were not successful in receiving funding, before and after their applications are depicted in Table 3. Data from Allied Health Scholars in KRESCENT were excluded from this analysis because of low numbers of trainees. Compared with nonfunded KRESCENT applicants, KRESCENT trainees (Post-Doctoral Fellows and New Investigators) had increases in average numbers of publications, average numbers of authors per manuscript, average number of addresses linked to authors on manuscripts, and percentage of manuscripts involving international collaborations. For Post-Doctoral KRESCENT awardees, both ARC and ARIF increased after training, compared with nonfunded applicants. While the ARC increased for KRESCENT New Investigators after training (1.68-1.88), ARIF decreased somewhat (1.59-1.46). Examples of high-impact first-author publications supported by KRESCENT that have improved diagnosis and management of kidney disease include a large registry cohort study that demonstrated an increased risk of adverse renal outcomes (including end-stage renal disease) with even a single kidney stone episode,[6] development and validation of a predictive model for progression of CKD (which has been adopted internationally),[7,8] and identification of a novel gene mutation causing atypical hemolytic-uremic syndrome.[9]

Table 3.

Bibliometric Analysis of KRESCENT Trainees Compared with Nonfunded Applicants.

Program	KRESCENT			Nonfunded applicants
	Before	After	Change	Before	After	Change
Average annual number of manuscripts
New Investigator	1.02	3.65	(+2.63)	0.80	2.86	(+2.06)
Post-Doctoral Fellowship	0.40	2.02	(+1.62)	0.41	1.41	(+1.00)
Average number of authors per manuscript
New Investigator	6.78	8.62	(+1.84)	6.81	7.65	(+0.84)
Post-Doctoral Fellowship	6.86	8.55	(+1.69)	7.12	7.92	(+0.80)
Average number of addresses per manuscript
New Investigator	3.63	5.78	(+2.15)	3.58	4.97	(+1.39)
Post-Doctoral Fellowship	3.37	5.56	(+2.19)	3.66	5.39	(+1.73)
% international collaboration
New Investigator	34.9	47.5	(+12.6)	30.3	40.1	(+9.8)
Post-Doctoral Fellowship	41.1	45.2	(+4.1)	38.3	33.7	(−4.6)
Average field-normalized citation rates (ARC)
New Investigator	1.68	1.88	(+0.20)	1.56	1.77	(+0.21)
Post-Doctoral Fellowship	1.18	2.26	(+1.08)	1.34	1.49	(+0.15)
Average field-normalized impact factors (ARIF)
New Investigator	1.59	1.46	(−0.13)	1.37	1.54	(+0.17)
Post-Doctoral Fellowship	1.28	1.59	(+0.31)	1.36	1.36	(0.00)

Note. Summative data for New Investigators (within 3 years of first academic appointment) and Post-Doctoral Fellows at the time of application to KRESCENT (Before), and after that time, until 2015 (After). Values in parentheses represent changes (“After” minus “Before”). KRESCENT = Kidney Research Scientist Core Education and National Training; ARC = average of relative citation; ARIF = average of relative impact factor.

Discussion

In 1999, the KFOC organized a national conference with its stakeholders to map out a long-term strategy for kidney research in Canada. This conference, Horizons 2000, established as a priority the enhancement of kidney research capacity in Canada,[1] which led to the development of the KRESCENT Program. The results of this review reveal several positive impacts of the program, including high rates of success in obtaining subsequent peer review operating grants and academic appointments at the level of Assistant Professor, enhanced numbers and quality of publications, and evidence for team-building and international collaboration.

A recent series of articles in the Canadian Journal of Kidney Health and Disease has focused on the serious challenges currently faced by kidney investigators in Canada in establishing independent research programs, obtaining peer review grant support, and engaging in KT.[10-14] The results of our survey indicate that a high proportion of KRESCENT trainees were successful in receiving peer review operating grant support from CIHR as either a PI or co-investigator (79%), and the majority of trainees acknowledged the importance of KRESCENT in achieving this degree of success. These data are particularly impressive in view of the decline in recent years in success rates for CIHR operating grant funding.[15]

A high proportion of Post-Doctoral Fellows who completed KRESCENT training were successful in obtaining academic appointments (76%), with the vast majority finding positions at academic institutions in Canada. This extent of success is notable, especially when one considers recent calls for reductions in training positions for PhDs and Post-Doctoral Fellows in the United States, due to a bottleneck in available academic appointments.[16,17] However, a recent CSN workforce survey reported that Nephrology Division Heads across Canada were seeking 14 full-time academic nephrologists immediately (defined as individuals with at least 75% of time devoted to teaching/research-related activities), and a total of 73 to 75 academic nephrologists are sought over the next 5 years.[18] Thus, young kidney research trainees (MDs and/or PhDs) in Canada have a unique opportunity for career development as independent investigators, and KRESCENT appears poised to serve this need.

Interdisciplinary research and team work have been recently emphasized as critical to scientific progress, especially in the health sciences.[19,20] Our survey data indicate that since completion of training, the majority of KRESCENT awardees conduct research as part of a multidisciplinary team (82%), and significant numbers are involved in KT1-, KT2-, or KT3-level translational research. The bibliometric data revealed that KRESCENT trainees have increased numbers of authors on manuscripts after completion of training, and increased international collaborations, compared with unsuccessful applicants. Furthermore, as shown for team-building and collaboration within the program, survey data indicated that trainees collaborate in publishing manuscripts and obtaining peer review grants together after completion of KRESCENT. The data therefore support an increased likelihood for KRESCENT-supported research findings to be widely disseminated.

The bibliometric analysis demonstrated that KRESCENT trainees at the New Investigator or Post-Doctoral Fellow level had substantial increases in research productivity upon completion of their training (as determined by the average annual number of manuscripts), compared with nonfunded applicants to the program. Similarly, ARC analysis showed that articles published by KRESCENT trainees received more citations than those published by nonfunded applicants, suggesting a higher level of scientific impact. In this regard, after KRESCENT training, the overall ARC for New Investigators was 1.88, and the ARC for Post-Doctoral Fellows was 2.26, values which compare favorably with the ARC for Program-supported articles arising from trainees in the Canadian Association of Gastroenterology (CAG)/CIHR/Partners training program between 2003 and 2011 (1.73).[4]

The results for ARIF reveal that KRESCENT Post-Doctoral Fellows published in journals of higher scientific impact after completion of training, compared with nonfunded applicants, and the average normalized impact factors were comparable with those observed for the CAG/CIHR/Partners training program.[4] However, the ARIF for KRESCENT New Investigators did not increase compared with values before KRESCENT training or compared with nonfunded applicants, nor was the increase in ARC higher than that observed with nonfunded applicants (0.20 vs. 0.21, respectively). Although reasons for this observation are unclear, it is notable that the average normalized ARIF prior to KRESCENT training for New Investigators was relatively high (1.59), as was the ARC (1.68), supporting a high performance level of KRESCENT awardees at baseline.

Perhaps the most revealing data in our analysis derive from the narrative comments obtained from the survey questionnaire. Many survey respondents highlighted the value of collaboration and interaction with peers across Canada. The responses support the importance of face-to-face workshop meetings in research training, which foster networking and team-building. Unfavorable comments regarding KRESCENT were few and largely restricted to the workload burden arising from the core curriculum. In this regard, work assignments in KRESCENT have been modified extensively since 2005, with reduction in time commitments and introduction of exercises with distinct potential benefits (including publication of review manuscripts, and mock grant- and manuscript-review panels).

Our analysis of KRESCENT has certain limitations. Although the survey questionnaire completion rate was significant (74%), those trainees who did not respond may have differing views on the strengths and weaknesses of the program. Data collected from trainees’ CVs are subject to potential error due to self-reporting, and a verification process was not conducted. The subjective nature of some of the survey responses can also be viewed as a limitation. With regard to the bibliometric data, we did not perform statistical comparisons, because use of tests of statistical significance is considered problematic and potentially detrimental in bibliometric analyses, as the data do not meet most assumptions for performance of these tests.[21] Finally, the use of data from nonfunded applicants to KRESCENT as a comparator group may be viewed as a weakness, because it might be expected that KRESCENT trainees should outperform this group on a number of research metrics. Nonetheless, our analysis is strengthened by the relatively high response rate to the questionnaire, and the robust bibliometric analysis, which allowed for comparison of research metrics before and after KRESCENT training.

The bibliometric data indicate that Canada’s position within the international kidney research community has strengthened in the period from 2009 to 2014, compared with 2000-2005, in terms of both impact and numbers of publications. Although the increase in the number of Canadian kidney-related publications and the ARC correspond temporally to the period when the KRESCENT Program was launched and implemented, the evaluation methodology does not permit direct attribution. Besides the potential contribution of KRESCENT, several factors likely played important roles in enhancing Canada’s position in the period between 2000 and 2014, including an increase in the overall number of funded Canadian kidney researchers.

Conclusions

From this evaluation, we conclude that the KRESCENT Program has enhanced kidney research capacity in Canada and fostered collaboration and KT. The program has resulted in increased numbers and quality of research publications, and promoted the development of a collaborative community of leading kidney research investigators. Our data suggest that to meet the needs of the kidney research community and stakeholders in Canada, KRESCENT should be sustained via long-term funding partnerships.