The importance of motivation in selecting undergraduate medical students for extracurricular research programmes.

INTRODUCTION: Extracurricular research programmes (ERPs) may contribute to reducing the current shortage in physician-scientists, but usually select students based on grades only. The question arises if students should be selected based on their motivation, regardless of their previous academic performance. Focusing on grades and lacking to take motivation into account when selecting students for ERPs might exclude an important target group when aiming to cultivate future physician-scientists. Therefore, this study compared ERP students with lower and higher previous academic performance on subsequent academic performance, ERP performance, and motivational factors.
METHODS: Prospective cohort study with undergraduate medical students who filled in a yearly questionnaire on motivational factors. Two student groups participating in an ERP were compared: students with first-year grade point average (GPA) ≥7 versus <7 on a 10-point grading scale. Linear and logistic regressions analyses were used to compare groups on subsequent academic performance (i.e. third-year GPA, in-time bachelor completion), ERP performance (i.e. drop-out, number of credits), and motivational factors (i.e. intrinsic motivation for research, research self-efficacy beliefs, perceptions of research, curiosity), while adjusting for gender and motivational factors at baseline.
RESULTS: The <7 group had significantly lower third-year GPA, and significantly higher odds for ERP drop-out than the ≥7 group. However, there was no significant between-group difference on in-time bachelor completion and the <7 group was not inferior to the ≥7 group in terms of intrinsic motivation for research, perceptions of research, and curiosity.
CONCLUSIONS: Since intrinsic motivation for research, perceptions of research, and curiosity are prerequisites of future research involvement, it seems beneficial to focus on motivation when selecting students for ERPS, allowing students with lower current academic performance to participate in ERPs as well.

Introduction

Serious concerns have been raised regarding the future of academic medicine in the past decades, as a result of a continued physician-scientist shortage [1-4]. Physician-scientists are health care professionals devoting a substantial amount of their time to both clinical care and research [5, 6]. Thereby, they are pivotal to bridge the gap between science and practice: they are important for both identifying relevant clinical problems to translate into research (i.e. bedside-to-bench) as well as translating research outcomes into clinical practice (i.e. bench-to-bedside) [6-10]. However, the decline in physician-scientists persists and the current workforce is aging [1–4, 10, 11].

A possible solution to retain the physician-scientist workforce could be to engage medical students in research early on in medical training [1, 12–14]. Engagement of undergraduate students in research could help to 1) promote awareness and critical appraisal of research among students, 2) motivate students to conduct research, 3) identify possibilities for a research career among students, and 4) recognize research talent by educators, researchers or physicians [9, 15–17].

The importance of research during medical training has been underlined in many medical educational frameworks and accrediting bodies [18-20]. Furthermore, the Boyer Commission presented a report with ten recommendations for reconstructing undergraduate medical education, with the first recommendation urging to make research-based learning the standard. As a result of the importance given to research by educational frameworks and accrediting bodies, and in line with the Boyer Commission’s call to promote engagement of undergraduate students in research, many medical schools started intra- and/or extracurricular initiatives to engage students in research [14, 21, 22]. As involvement in research during medical training is associated with involvement in research during future professional practice, stimulating undergraduate students to engage in research during early phases of medical training could well be seen as a first step in the physician-scientist pipeline [12, 23, 24].

In order to promote in-depth research involvement, an academic mindset, and by extension cultivate future physician-scientists, many medical schools implemented extracurricular research programmes (ERPs) [1, 22]. Such programmes occur under different names and diverse formats, e.g. MD/PhD programmes, capstone programmes, summer research programmes, and Honours programmes [13, 22, 25, 26]. Some previous studies into the effects of ERPs showed that they enhanced students’ interest in and appreciation of research, increased research skills and productivity, and promoted continued involvement in research [1, 27–30].

Selection procedures for ERPs, especially Honours programmes, usually focus on grades as a means to select ‘excellent’ students [31-34]. Ericsson (1993) defines excellence by current performance; perceiving students who have the highest grades as excellent students [35, 36]. When defining excellence in this manner, selection of students for ERPs based on grades seems logical.

However, although high grades may be predictive for knowledge recall, they lack predictive validity for knowledge application and higher order cognitive skills [37]. Conducting research can be seen as a higher order cognitive skill, which implies that selecting students for ERPs should go beyond just grades. This may certainly be the case in medical Bachelor degree programmes, in which most of the grades are based on exams that focus mainly on knowledge recall. Moreover, the goals of the specific ERPs should be taken into account as well. If the pre-eminent goal of ERPs is to develop an academic mindset and cultivate future physician-scientists, it seems questionable to focus solely on grades. Additionally, it is important to keep in mind that grades do not necessarily reflect all the competencies that are valued in the job market. This might be especially the case for health care professionals who must be able to take on different roles (e.g. communicator, collaborator, health advocate, scholar) [19].

Indeed, an alternative perspective on excellence defines excellent students in terms of potential performance, emphasizing the equal importance of motivation next to above-average intellectual ability [38]. This perspective, focusing on motivation as a parameter for excellence as well, might better align with the goal of ERPs to promote research involvement, an academic mindset or even cultivate physician-scientists.

Perceiving students with the highest grades as excellent students, without taking motivation into account when selecting students for ERPs that have the goal to cultivate physician-scientists, might exclude a very important target group of students that are motivated for research and willing to pursue a research-oriented career. In fact, according to Weaver and colleagues, the strongest predictor for a physician-scientist career is indeed an existing passion for research [12]. Previous research also showed that, in line with the Self-Determination Theory, intrinsic motivation for research is related to (further) research involvement in medical school, which in turn is related to research involvement in future professional practice [23, 39]. In addition, in line with the Social Cognitive Theory, research self-efficacy is believed to be related to research motivation and the tendency to conduct research [40]. Furthermore, according to the Theory of Planned Behaviour, perceptions are related to intentions, which in turn are related to the desired behaviour [41]. Lastly, curiosity is identified as an important antecedent for conducting research [23]. In sum, as the main goal of many ERPs is to foster future physician-scientists, intrinsic motivation for research, research self-efficacy beliefs, positive perceptions of research, and curiosity might be valuable objectives to pursue and promote among ERP students.

Students with lower current academic performance might become equally motivated for research as a result of participating within the ERP, thereby contributing to the pool of future physician-scientists. In other words, the question arises if emphasis should shift from grades towards motivation when selecting for ERPs aiming to cultivate future physician-scientists. Without taking motivation into account, an important target group might be excluded.

Therefore, the aim of this study was to investigate if two groups of students in an ERP (students with higher versus lower previous academic performance) differ in subsequent academic performance (i.e. third-year GPA, in-time bachelor completion), ERP performance (i.e. drop-out, number of credits), and motivational factors (i.e. intrinsic motivation for research, research self-efficacy beliefs, perceptions of research, curiosity), by using a prospective, longitudinal approach with a baseline measure. First, we hypothesized that the higher previous performance group will outperform the lower previous performance group on subsequent academic and ERP performance. Second, we hypothesized that the higher previous performance group will not outperform the lower previous performance group on motivational factors because academic skills do not by definition affect motivation for research and students with lower grades may still have a passion for research.

Materials and methods

Context

Within the Netherlands, eight universities provide medical education in line with the Dutch National Blueprint for Medical Education, which is based on the CanMEDS [18, 19]. Medical education in all universities is comparable in structure consisting of a six-year undergraduate educational programme, with a three-year programme leading to a Bachelor degree and a subsequent three-year programme leading to a Master degree in Medicine. Leiden University Medical Center (LUMC) is one of eight medical faculties in the Netherlands. Medical students’ academic performances are assessed with grades on a 10-point grading scale. Within this grading system, 10 is the highest achievable grade and 6 is the pass grade. GPA is the average of all obtained grades, also reported on a scale of 1 to 10. Grades are not only used to assess students’ academic performance, but in general also play a role in selecting students for ERPs. In most ERPs, a GPA of 7 is the threshold for entering. However, on top of the core curriculum, LUMC offers students the possibility to participate in a voluntary ERP (i.e. research-based Honours programme), without requiring a GPA of 7 or higher as the threshold for entering. Every medical student can apply, as in the past years selection was mainly based on self-selection without very strict institutional criteria. Consequently, the way our programme is implemented offers a unique opportunity to compare students with a GPA below and above 7. Students do, however, need to graduate in time and with a GPA of 7 or higher within the regular educational programme in order to receive a certificate from the ERP. Within this programme, LUMC aims to foster research talent and students are provided with the opportunity to conduct research individually. The programme starts in the second year of medical school and lasts two years. On average, about 50–60 motivated students participate in the programme each year, which represents 15–20% of the whole second year cohort of medical students. Furthermore, students need to obtain 30 credits (ECTS, i.e. European Credit Transfer and Accumulation System, which means that students have to invest 28 hours of active study per credit) [26].

Design and participants

This prospective cohort study is part of a longitudinal study in which one cohort of medical undergraduates is followed through medical education. All students starting medical school in 2016 were asked to participate in the longitudinal study, and requested to fill in one questionnaire each year (e.g. November 2016, January 2018, December 2018). Furthermore, grades and ERP performance characteristics were obtained. In the present study, all students participating in the ERP of the LUMC were included.

Materials and definitions

To investigate the effect of student group on academic performance, GPA of the third year of medical education (GPA3) and time to degree were drawn from university files. To investigate the effect on ERP performance, drop-out from and number of ECTS in the programme were drawn from university files as well. Lastly, to examine the effect of student group on motivational factors, questionnaire data were used [39]. The questionnaire was based on existing and validated scales, which were adjusted to the medical education setting with a focus on conducting research. Students were asked to score items on a 7-point Likert scale ranging from 1 –‘totally disagree’ to 7 –‘totally agree’.

Intrinsic motivation for research was defined as students being motivated to conduct research out of their own interest. The scale consisted of five items (e.g. ‘doing research is fun’) based on the Interest/Enjoyment Scale of the Self-Determination Questionnaires [42, 43]. Research self-efficacy was defined as students’ beliefs about their ability to conduct research. The scale consisted of three items (e.g. ‘I feel I am competent enough to do research’) and was self-developed, but inspired by the Dutch General Self-Efficacy Scale and the Academic Efficacy Scale [44, 45]. Perceptions of research were defined as students’ beliefs about the value of research. The scale consisted of five items (e.g. ‘It is important for medical professionals to have scientific skills’) of the Student Perception of Research Integration Questionnaire [46]. Lastly, curiosity was measured with ten items (e.g. ‘I enjoy investigating new ideas’) of the Epistemic Curiosity Scale [47].

Procedure

After adjustment of the existing scales, the questionnaire was translated from English to Dutch by using the forward and backward translation procedure. In a pilot study, we pretested the questionnaire amongst ten undergraduate medical students in their second year of medical education, after which two minor adjustment to two items were made. All first-year medical students starting medical training in 2016 were approached by the first author during a workgroup session (T1 baseline—November 2016). The same students were approached again in the first semester of their second (T2—January 2018) and third year (T3—December 2018) of medical school.

Students were informed about the goals and voluntary nature of participating in this study. Additionally, it was explained to students that all data would be used for research purposes and would be processed anonymously. Furthermore, written consent was asked to connect data of all questionnaires and to connect questionnaire data to prior and subsequent academic and ERP performance. This study was approved by the ethical review board of the Netherlands Association of Medical Education: reference number 952.

Analyses

We used descriptive statistics to report demographics of the participants. We calculated Cronbach’s alpha to estimate the reliability of the scales. Mean scores were calculated for the motivational factors. We established normal distribution of the data. Students in the ERP were divided in two groups based on GPA of the first year of undergraduate medical education (GPA1) prior to the start of the ERP: GPA1 ≥7 versus GPA1 <7. We used independent t-tests to examine if the two groups differed on the motivational factors at the start of medical training (i.e. T1 –baseline scores). Both univariate as well as multivariate logistic and linear regression analyses were used to compare the two groups of students on academic performance, ERP performance, and motivational outcomes, adjusting for baseline motivation and gender.

To test our first hypothesis that the ≥7 group outperforms the <7 group on subsequent academic and ERP performance, we assessed whether the difference in performance was significantly different from zero by looking at 95% confidence intervals. However, to test our second hypothesis that the ≥7 group does not outperform the <7 group on motivational outcomes, we need a different approach. More specifically, we sought to demonstrate that there was no difference in motivational outcomes between the two groups. However, testing whether a difference in motivational outcomes is significantly different from zero will not help in demonstrating there is in fact no difference, as lack of statistical significance would not prove absence of difference between the two groups. Instead, we assessed whether the difference was not more than a certain pre-set margin. In this so-called non-inferiority design, the non-inferiority margin is the maximum difference below which we consider the groups to be not meaningfully different [48]. We elaborated on the non-inferiority margin and reached consensus on a non-inferiority margin of 0.5 on the motivational scales. Hence, we tested if the groups differ by less than 0.5 on motivational outcomes, and thus that the difference is significantly smaller than 0.5, by assessing whether 0.5 is outside the 95% confidence interval. If so, we can conclude that the <7 group and the ≥7 group do not meaningfully differ in motivational outcomes, and the <7 group at most performs only marginally worse. We analysed all data using IBM SPSS Statistics version 23.

Results

Within this cohort existing out of 315 students, a total of 59 students participated in the ERP. All 59 students consented to participate in the current study, of whom 13 (22%) were male and 46 (88%) were female students. This male/female distribution is comparable to the distribution within the whole cohort of medical students. The 59 students were divided in 29 students in the ≥7 group (49.2%) and 30 students (50.8%) in the <7 group. Baseline scores of the two groups on GPA1, intrinsic motivation for research, research self-efficacy, perceptions of research, and curiosity can be found in Table 1. All students were included in the analyses of academic and ERP performance. In total, 57 out of 59 ERP students participated in the baseline survey (96.6%), and 54 out of 59 students participated in the third-year survey (91.5%) addressing the motivational factors. Cronbach’s alpha was calculated for the scales of the questionnaire and ranged from .81 to .88 at baseline T1 (November 2016, first year of medical school) and from .80 to .89 at T3 (December 2018, third year of medical school). At baseline, the two groups of students did not differ significantly on the motivational factors. GPA3, in-time bachelor completion, drop-out rates, number of credits within the programme and T3 motivational scores are reported in Table 2.

Table 1

Baseline characteristics divided by ≥7 (n = 29) and <7 (n = 30) student group.

	≥7 student group	<7 student group
Gender
Male	6 (20.7%)	7 (23.3%)
Female	23 (79.3%)	23 (76.7%)
GPA year 1
n	29	30
M (SD)	7.64 (.44)	6.73 (.15)
Min-Max	7.02–8.92	6.41–6.96
Intrinsic motivation T1
n	28	29
M (SD)	5.98 (.64)	5.72 (.71)
Min-Max	4.8–7.0	4.2–6.8
Research self-efficacy T1
n	28	29
M (SD)	5.13 (.98)	5.14 (.96)
Min-Max	3.0–7.0	3.0–6.7
Perceptions of research T1
n	28	29
M (SD)	5.92 (.73)	5.81 (.84)
Min-Max	3.8–7.0	2.8–6.8
Curiosity T1
n	28	29
M (SD)	5.45 (.65)	5.47 (.73)
Min-Max	4.1–6.6	4.3–6.9

Table 2

Overview of outcome measures divided by ≥7 (n = 29) and <7 (n = 30) student group.

	≥7 student group	<7 student group
GPA year 3
M (SD)	7.62 (.41)	7.14 (.25)
Min-Max	6.77–8.65	6.70–7.75
In-time bachelor completion
no (%)	4 (13.8%)	5 (16.7%)
yes (%)	25 (86.2%)	25 (83.3%)
ERP drop-out
no (%)	18 (62.1%)	9 (30%)
yes (%)	11 (37.9%)	21 (70%)
Amount of ECTS
M (SD)	23.72 (17.39)	12.17 (13.29)
Min-Max	0–55	0–41
Intrinsic motivation T3
n	26	28
M (SD)	5.98 (.61)	5.81 (.69)
Min-Max	4.8–7.0	4.0–7.0
Research self-efficacy T3
n	26	28
M (SD)	5.10 (.98)	4.70 (.81)
Min-Max	3.3–7.0	2.0–6.0
Perceptions of research T3
n	26	28
M (SD)	5.45 (.79)	5.75 (.87)
Min-Max	4.0–7.0	3.8–7.0
Curiosity T3
n	26	28
M (SD)	5.29 (.64)	5.49 (.69)
Min-Max	3.8–6.7	4.1–6.9

Academic performance

An effect of student group on GPA3 was found, with students in the <7 group performing significantly lower in the third year of medical education (β = -.48, 95%CI = -.66 - -.29). This effect remained after adjusting for gender and the motivational factors at baseline (β = -.46, 95%CI = -.67- -.25). However, there was no effect on in-time bachelor completion (crude OR = .80, 95% CI = .19–3.33; adjusted OR = .83, 95%CI = .17–4.00).

ERP performance

With regard to ERP performance, a significant effect was found from student group on ERP drop-out. The odds for ERP drop-out were significantly higher in the <7 group (OR = 3.82, 95% CI = 1.29–11.28), also after adjusting for gender and motivational baseline scores (OR = 4.25, 95% CI = 1.29–13.94). Furthermore, a significant effect was found regarding the number of credits in the programme, with students in the <7 group obtaining less credits than students in the ≥7 group (crude β = -11.55, 95%CI = -19.60 - -3.50; adjusted β = -12.52, 95%CI = -20.83 - -4.20).

Motivational factors

With regard to the motivational factors, the non-inferiority margin was set at 0.5 points. Our findings showed that, after adjusting for gender and the motivational factors at baseline, -0.5 was not in the 95% confidence interval for intrinsic motivation for research (β = -.13, 95%CI = -.44 - .19), perceptions of research (β = .29, 95%CI = -.16 - .74), and curiosity (β = .12, 95%CI = -.21 - .44). Thus, with 95% confidence, the difference in these motivational factors is smaller than 0.5 and students in the <7 group were not inferior to ≥7 group when it comes to these motivational factors. When looking at research self-efficacy, -0.5 is within the confidence interval (β = -.40, 95%CI = -.89 - .09). Therefore, we cannot conclude that the <7 group was not inferior to the ≥7 group. An overview of these findings can be found in Table 3.

Table 3

Regression model of the effect of type of student on the motivational factors in the third year of medical education.

	Crude	Adjusted for gender and motivational baseline scores
	β (95%CI)	β (95%CI)
Intrinsic Motivation	-.16 (-.52 - .19)	-.13 (-.44 - .19)
Research self-efficacy	-.40 (-.89 - .09)	-.40 (-.89 - .09)
Perceptions of research	.30 (-.16 - .75)	.29 (-.16 - .74)
Curiosity	.20 (-.16 - .56)	.12 (-.21 - .44)

*reference: ≥7 group

Discussion

Within this study, we compared two groups of students on three outcome levels: academic performance, ERP performance, and motivational factors. We hypothesized that the ≥7 group would outperform the <7 group on academic and ERP performance, but not on motivational factors. In line with our first hypothesis, the <7 group had lower academic performance (GPA3), significantly higher odds for ERP drop-out and less credits within the ERP compared to the ≥7 group. Confirming our second hypothesis, the <7 group is not inferior to the ≥7 group on intrinsic motivation for research, perceptions of research, and curiosity in the third year of medical education. In other words, intrinsic motivation for research, perceptions of research, and curiosity in the third year of medical education did not differ meaningfully between both groups. The only contradiction to our hypotheses was found on in-time bachelor completion, as the two groups of students did not significantly differ in obtaining their bachelor degree in the appointed amount of time.

In line with our hypotheses, the <7 group obtained lower levels in GPA3, but did however not seem to differ in time to obtain their degree. ERPs expose students to additional workload on top of their regular medical training [34, 49]. One concern when it comes to including students beyond the ‘excellent’ and ‘high-achieving’ student population in certain programmes, is that these students might not be able to combine the additional workload with the regular courses, and that ERP participation will have a negative impact on academic performance in the regular programme. As we did not find a significant difference in in-time bachelor completion, it could be that the <7 group, though on average obtaining 0.5 points lower with regard to GPA3, did not differ from the ≥7 group on in-time bachelor completion. Moreover, the mean difference between the two groups on GPA3 narrowed as compared to GPA in the first year of medical school, as the GPA of the <7 group increased to a larger extent. These findings indicate that ERP participation is not at the expense of the regular programme. ERP participation might even lead to a greater advantage for the <7 group, as participating in the programme may even enhance their GPA in the subsequent years. An explanation for this could be that students in the ERP are surrounded by highly motivated peers [49]. This is in line with the ‘reflected glory effect’, referring to the tendency individuals have to relate one’s self-perceived ability to the success of others [50]. Within this context, students in the <7 group might identify themselves with the selective group of high-achieving and motivated peers, which has a positive impact on their self-perceived ability. From this perspective, the presence of the ≥7 group is a positive factor for the performance of the <7 group. This, in turn, is in line with findings by another study showing that improved self-concept is related to increased learning outcomes [51].

Our findings suggest that the <7 student group obtained significantly less credits within the programme. This is probably associated with the fact that for students in the <7 group the odds for ERP drop-out were about four times as high. It is remarkable that, though comparable in motivation for research, the drop-out in the <7 group is higher as compared to the ≥7 group, possibly implying that motivation does not lead to ERP completion. The question arises if attrition from the ERP results from a lack of ability to conduct research, or that other reasons might lead to the decision to quit the programme among students. Dropping out of the programme might not by definition mean that students are deterred from research. A reason for the higher ERP drop-out rate may be that, although students with GPA <7 are allowed to participate within the programme, a requirement is that students graduate and receive their medical Bachelor’s degree with an average grade of 7 or higher to obtain the ERP certificate [26]. Students who already started the programme with lower grades might feel they will not meet this requirement, and could therefore decide to quit the programme in advance. In addition, this rule sends out the implicit message that students scoring below 7 are not the type of students that are supposed to be enrolled within such ERPs [52]. Some students within the regular cohort also voluntarily conduct extracurricular research without following the structured ERP, so it could be that students dropping out from the ERP decide to follow this path as well. Another possible explanation for the higher chance of dropping out in the <7 group may be the ‘big fish little pond–effect’ [50], which has the opposite effect for other types of students as compared to the reflected glory hypothesis. According to the big fish little pond–effect, students’ self-perceived ability is determined by the comparison with peers. Students participating in an ERP compare themselves with the smaller group of participants within the programme, while largely surrounded by high-achieving, ‘top of their class’ peers. A similar student in the regular programme will compare itself with the bigger pond of students, differing in cognitive ability. As a result of this change in reference, the <7 group within the programme might have lower levels of self-perceived ability because they are surrounded by some ‘big fish’ (i.e. the high-achieving GPA1 ≥7 students) in their little pond (i.e. smaller group of programme participants) [50, 53, 54].

Should this higher level of drop-out, then, be a reason to only include ‘excellent’ students in certain programmes in the future? From an efficiency perspective, one might say that drop-out within a rather costly programme should be avoided. Furthermore, it could be argued that select groups of students receive additional education which needs to be justified, especially because these graduates are more appealing for job recruiters [53]. Lastly, one might wonder if students not completing the ERP will be able to deal with the pressures in future professional practice, for instance combining research and clinical duties. These perspectives might contribute to the idea of solely including high-achieving students in such extracurricular programmes to prevent attrition, aligned with Ericsson’s perspective on ‘excellence’.

However, when evaluating ERPs, a focus on academic and ERP performance might provide an incomplete image when aiming to deliver professionals who fit the needs of the specific field [33, 53]. When looking at the motivational factors in the current study, our results are inconclusive with respect to research self-efficacy beliefs in the third year of medical education, although they seem to be somewhat higher in the ≥7 group. A study by Kool and colleagues [55] showed that high-achieving students were more performance oriented, defined as students’ pursuit to outperform peers and show their own abilities, which might explain the higher levels of confidence in their own abilities among the students in the ≥7 group. In addition, for some students the big fish little pond–effect might apply here as well. A practical implication derived from these findings might be to support motivated, above-average ability students in ERPs in such ways that their research self-efficacy beliefs are enhanced, as research self-efficacy is related to research motivation and the tendency to conduct research [40].

More importantly, our study showed that the two groups of students are comparable in intrinsic motivation for research, perceptions of research, and curiosity in the third year of medical education. In line with major theories like Self-Determination Theory, Social Cognitive Theory, and Theory of Planned Behaviour, as well as findings from previous studies, these constructs are related to actual research engagement [23, 39–43].

Thus, despite the higher ERP drop-out rates in the <7 group, the group is not inferior to the ≥7 group on the desired outcomes that are imperative to cultivate future physician-scientists. In addition, one might say these students are supported in their development regardless of whether they eventually finished the ERP. But above all, if the pre-eminent goal of ERPs is to develop future physician-scientists, this goal is not endangered by including students with lower academic performances in their first year of medical training. In fact, these students might well belong in the target group when aiming to cultivate future physician-scientists and selection based solely on grades poses the risk to exclude a motivated group of students from the physician-scientist training pipeline.

To summarize, students in the <7 group quit the ERP more often and have lower GPA in the third year of medical education, but ERP participation may help to enhance student GPA of the first years of undergraduate medical study in the <7 group. More importantly, the <7 group scored comparable to the ≥7 group on intrinsic motivation for research, perceptions of research and curiosity, which are all motivational factors underlying research involvement in future professional practice. Therefore, when aiming to cultivate future physician-scientists, our findings imply that the perspective on excellence emphasizing potential performance and the equal importance of motivation is more aligned with the aims of ERPs. Especially when taking into account that medical students invest a great amount of academic effort before entering medical school and are selected on, among others, cognitive abilities [56]. To conclude, this could mean that, in order to use ERPs as a step in the physician-scientist pipeline, motivation should be given importance in selecting students for ERPs, allowing students with a GPA lower than seven to participate within such programmes as well. This could be established by using a selection model in which GPA is not perceived as a threshold to enter the ERP. Insights in motivation of students willing to self-select within the ERP could be elucidated by, for instance, asking students to write a motivation letter to reflect on their feelings of competence within the regular educational programme (i.e. academic self-efficacy) and their motivation to participate within the specific ERP. Furthermore, it could be valuable to offer students a low-threshold activity in which they can get acquainted with the ERP to substantiate their willingness to participate within the ERP.

Strengths, limitations and future research

First, our study was performed within a single institute. However, our medical curriculum is comparable to other medical curricula as the educational programme is based on the Dutch National Blueprint for Medical Education, which in turn is aligned with, among others, the CanMEDS [18, 19]. Additionally, many medical schools worldwide provide undergraduate students with ERPs. Second, the outcome measures of the current study are not long-term measures like, for instance, publication rates. However, previous studies have shown that both research involvement within medical training, as well as the measured motivational factors within this study, are related to long-term scientific involvement [23, 24, 39]. Valuable for future research might be to include long-term effects with scholarly output (e.g. publications and conference contributions), and a career as a physician-scientist. Third, the groups within our study were relatively small. Therefore, an interesting future research avenue might be to conduct this study within different contexts and, when possible, larger groups. Furthermore, future research could focus on identifying causes of lower credits in the <7 group, as well as investigating if ERP drop-outs perceive this negatively and what is needed to support all different types of students and promote every student to flourish within such ERPs. A qualitative approach could help to provide more in-depth insights into the above mentioned topics. In addition, reasons for drop-out and subsequent intentions to pursue research, or lack thereof, might be valuable to identify.

Conclusion

Two groups of students within an ERP were compared on three outcome levels: academic performance, ERP performance, and motivational factors. The <7 group obtained lower levels of GPA3 and had significantly higher odds for ERP drop-out. On the contrary, the <7 group did not differ from the ≥7 group on in-time bachelor completion, and had comparable levels of intrinsic motivation for research, perceptions of research, and curiosity in the third year of medical education, which are all factors underlying research involvement in future professional practice. Therefore, for ERPs aiming to develop future physician-scientists, a shift from an emphasis solely on grades towards taking motivation into account could be beneficial for the selection for such programmes, allowing students with lower current performance to participate as well.

30 Apr 2021

PONE-D-21-11278

The importance of motivation in selecting undergraduate medical students for extracurricular research programmes

PLOS ONE

Dear Dr. Ommering,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Kindly find the reviewers' comments. My personal reading of the manuscript shows it is well-written and after the required changes can be an important contribution to the journal.

Please submit your revised manuscript by Jun 14 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Pathiyil Ravi Shankar

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

My commenst are:

Lines 80 and 81: The authors should describe the recommendations of the Boyer Commission for the benefit of readers.

How are students selected currently for the ERPs? Is only the GPA considered? How are students matched to different ERPs?

How exactly do the authors propose to measure motivation for research and other characteristics while selecting students for ERP? What model of student selection are they proposing and what weight will be provided to GPA and other factors including motivation?

The authors should also make available/deposit the data associated with the research in a publicly accessible data repository. Personal details can be deidentified if needed.

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1) Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2) Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is a well written manuscript. The data is interesting. It is a prospective cohort study with undergraduate medical students who filled in a yearly questionnaire on motivational factors.

Two medical student groups participating in an ERP: students with first-year grade point average (GPA) ≥7 versus <7 on a 10- point grading scale. 70% students in the GPA <7 group dropped out of ERP compared to 38% in the GPA >7 group. The lower GPA group loss had lower GPA at the end of the study.

In terms of questionnaires related intrinsic motivation for research, perceptions of research, and curiosity at the end of the study there was no difference between groups. The authors propose allowing students with lower current academic performance to participate in ERPs is beneficial.

The conclusion does not seem supported by the results. Despite answering the questions related to research motivation and curiosity similarly, the majority of the students in the lower GPA group did not complete the ERP program. The research curiosity and motivation did not translate into continuation / completion of ERP.

How did the select students who continued the ERP (9/30) in the lower GPA group score on the research perception questionnaires compared the students who continued the ERP in the higher GPA group?

It would be interesting to follow this cohort to see how many will become actual physician scientists.

Table 1 and 2: please insert a column for P value between the groups.

Reviewer #2: The hypothesis is relevant and the methodology is aligned with the objectives. Qualitative data gathering with factor analysis for the conclusions derived would have strengthened the conclusions of the study. The study would be more meaningful if an attempt was made to identify causes of lesser credits in <7 group inspite of comparable intrinsic motivation. If the students in <7 group dropped out to meet academic requirements , it would establish the counter argument that intrinsic motivation alone is not enough for participation in ERP. Authors could highlight regarding remedial measures offered, if any for the students in <7 group to clarify its role in lesser in group difference in GPA3 as compared to GPA 1.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Vinutha Shankar MS

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

11 Jun 2021

Dear Pathiyil Ravi Shankar,

We would like to thank you for the opportunity to revise our manuscript. In addition, we would like to thank you and the reviewers for the helpful review comments, which are greatly appreciated. The comments offer us guidance in the review process and help to improve our manuscript. Below we provide an overview on how we processed the review comments.

Editor comments

“Lines 80 and 81: The authors should describe the recommendations of the Boyer Commission for the benefit of readers.”

Done as requested (see revised manuscript, page 4; lines 80-84).

“How are students selected currently for the ERPs? Is only the GPA considered? How are students matched to different ERPs?”

Our study is conducted within Leiden University Medical Center (LUMC), which offers one extracurricular research programme for which every student can apply. That means that self-selection is the basis for enrolment within the single programme that the LUMC offers. There are no institutional selection criteria, so GPA is not considered to be a factor to deny students to participate within the programme. Students do, however, need to graduate in time and with a GPA of 7 or higher within the regular educational programme in order to receive a certificate from the ERP. This means that GPA is not considered as a selection requirement when entering the ERP, but within our current ERP it does play a role in order to receive a certificate. The self-selection approach to enter the ERP is not very common, as most ERPs select students based on GPA and regard a GPA of at least 7 (on a 10-point grading scale) to be sufficient to participate in an ERP. However, as the programme within our institute is accessible for every student, this did gave us the unique opportunity to compare students scoring above the 7 and below the 7, the latter being a motivated group of students that might want to pursue a research oriented career but in many other institutes and programmes would not have been given the chance to participate and act on their intrinsic motivation for research. In order to make the distinction between how students are generally selected for ERPs and our approach to letting students enrol by self-selection more apparent, within the revised manuscript on page 8 (line 179) we now explicitly mention that there is no GPA threshold for entering the ERP. Furthermore we structured the paragraph different within the revised manuscript, in such a way that first the general context of ERP selection is described, after which the way self-selection is handled within the LUMC immediately follows while also mentioning the requirements to obtain the ERP certificate (page 8, lines 177-184). We hope this clarifies the matter.

“How exactly do the authors propose to measure motivation for research and other characteristics while selecting students for ERP? What model of student selection are they proposing and what weight will be provided to GPA and other factors including motivation?”

In line with our vision, we strive towards a selection model in which GPA is not perceived as a threshold to enter the ERP. As previous studies showed that intrinsic motivation for research is related to actual research involvement in future professional practice, combined with studies showing that grades are not predictive for higher order cognitive skills, we might exclude potential future physician-scientists when denying motivated students access to the ERP just because their GPA is below 7. Our key message therefore is to offer motivated students the opportunity to participate within the ERP regardless of their academic GPA when starting the ERP, because these motivated students might be a target group when aiming to foster research talent and cultivate future physician-scientists (see page 6 and 7 of the manuscript). The finding that the <7 group is not inferior to the ≥7 group within the ERP corroborates this vision. We hope this clarifies that we do not propose to give a certain amount of weight to GPA and other factors, we believe in the self-selection model enabling students that are passionate about research and willing to enter or pursue in this pipeline. Within the revised manuscript, we now elaborate on how to obtain insights in motivation for research (page 21, line 449-456). We feel that students, for instance, could be asked to write a motivation letter reflecting on their feelings of competence within the regular educational programme (i.e. academic self-efficacy) and their motivation to participate within the specific ERP. Furthermore, we believe it could be valuable to offer students a low-threshold activity in which they can get acquainted with the ERP to substantiate their willingness to participate within the ERP. It could also be interesting, however, to measure motivation for research while selecting students for the ERP, for which we did develop a questionnaire. This could, for instance, help to provide insights into the development of motivation for research through the ERP.

“The authors should also make available/deposit the data associated with the research in a publicly accessible data repository. Personal details can be deidentified if needed.”

We agree that from an open access perspective data should be made available when possible. However, in the consent form, we informed the participating students that data would not be traceable to individual participants. As this is a somewhat smaller sample size from one educational year in one academic medical center in the Netherlands, anonymity of the participants could not be guaranteed when the data associated with the research is deposited in a publicly accessible data repository. We hope that the academic editor acknowledges that due to these ethical reasons we are not able to deposit are data. Of course, when approached by other researchers, we would be open and happy to collaborate with those interested.

Journal requirements

“Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming.”

We thoroughly looked at the Plos One submission guidelines and style templates to ensure that our manuscript meets the style requirements, which led to one minor adjustment on page 8 (line 164) of the manuscript: we replaced the level 1 heading ‘Methods’ by ‘Materials and methods’.

“Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.”

In the procedure-subheading at page 11 we already mentioned that students’ consent was asked to connect data of all questionnaires and to connect questionnaire data to prior and subsequent academic and ERP performance. However, within the revised manuscript we now explicitly mentioned that written consent was asked (see page 11, line 241 of the revised manuscript).

Reviewer 1

“This is a well written manuscript. The data is interesting. It is a prospective cohort study with undergraduate medical students who filled in a yearly questionnaire on motivational factors. Two medical student groups participating in an ERP: students with first-year grade point average (GPA) ≥7 versus <7 on a 10- point grading scale. 70% students in the GPA <7 group dropped out of ERP compared to 38% in the GPA >7 group. The lower GPA group loss had lower GPA at the end of the study. In terms of questionnaires related intrinsic motivation for research, perceptions of research, and curiosity at the end of the study there was no difference between groups. The authors propose allowing students with lower current academic performance to participate in ERPs is beneficial. The conclusion does not seem supported by the results. Despite answering the questions related to research motivation and curiosity similarly, the majority of the students in the lower GPA group did not complete the ERP program. The research curiosity and motivation did not translate into continuation / completion of ERP.”

Thank you for the compliments regarding our manuscript and for summarizing our findings correctly. Indeed, the drop-out rate is higher in the <7 group and their GPA is lower in year three as compared to the ≥7 group. However, the <7 group had lower levels of GPA to begin with and when looking at Table 1 and 2, it seems that their GPA increased to a larger extent then the GPA in the ≥7 group. One of the mentioned concerns when considering to let students below a certain threshold GPA apply and participate within an ERP is that it will negatively impact academic performance within the regular educational programme. Our findings imply that this is not the case and that ERP participation is not at the expense of academic performance within the regular programme for the students starting the programme with a GPA below 7. It might even lead to a greater advantage as the mean difference between the two groups decreases because of the greater increase in GPA in the <7 group (for our interpretation and explanation through the lens of the “reflected glory” effect, we kindly refer to page 17, lines 359-366 of our manuscript). We apologize for the confusion and within the revised manuscript we now explicitly mentioned that the GPA of the <7 group increased to a larger extent (page 17, lines 355-356). We hope this helps to clarify our message.

The reviewer comes with an interesting interpretation by stating that the majority of the students within the <7 group did not finish the programme, and that research curiosity and motivation did not translate into ERP completion. We agree with the reviewer that this could indeed be the case, however, we believe that the academic requirements held by our institution (i.e. students must graduate and receive their medical Bachelor’s degree with an average grade of 7 or higher to obtain the ERP certificate) result in students choosing to quit the programme in advance, while still being the curious and motivated student that started the programme. Especially, as some students communicate these type of reasons to the ERP-coordinator. In the revised manuscript we have also added the notion as mentioned by the reviewer that it could mean that curiosity and motivation alone is not enough (see page 18, lines 369-371 of the revised manuscript). We hope our future research will shed light on this particular issue. To conclude, we do feel the need to mention here that motivation on the long run feels more important than whether or not the student obtained the certificate. One third of the students in the <7 group did obtain their certificate and otherwise we would have missed out on this group. We do however plan to study in more detail why students drop out of the programme and how their motivation for research could be described after dropping out.

“How did the select students who continued the ERP (9/30) in the lower GPA group score on the research perception questionnaires compared the students who continued the ERP in the higher GPA group?”

When comparing these 18 (≥7 group) versus 9 (<7 group) students, our findings show that the main conclusions to be drawn from the results are comparable. When comparing these smaller groups our findings showed that, after adjusting for gender and the motivational factors at baseline, -0.5 was not in the 95% confidence interval for intrinsic motivation for research (β = .10, 95%CI = -.42 - .63), perceptions of research (β = .65, 95%CI = -.11 – 1.41), and curiosity (β = .24, 95%CI = -.44 - .93). Thus, with 95% confidence, the difference in these motivational factors is smaller than 0.5 and students in the <7 group are not inferior to ≥7 group when it comes to these motivational factors. When looking at research self-efficacy, -0.5 is within the confidence interval (β = -.58, 95%CI = -1.32 - .17). Therefore, we cannot conclude that the <7 group is not inferior to the ≥7 group. However, the groups within this analysis were very small so caution is needed when interpreting these results.

“It would be interesting to follow this cohort to see how many will become actual physician scientists.”

We absolutely agree with the reviewer that it would be interesting to follow this cohort to see how many students will become actual physician-scientists. Within our future research paragraph on page 22 (line 468) of the revised manuscript we have added ‘a career as a physician-scientist’ (next to scholarly output like publications and conference contributions) as a possible long-term effect that would be interesting for future research.

“Table 1 and 2: please insert a column for P value between the groups.”

Table 1 illustrates the descriptives of the participants within our group – as strictly speaking no formal hypothesis is tested here, we deliberately did not insert the p value. We believe that inserting the p value (or confidence intervals) would be informative when a research question is formed and consequently a hypothesis is tested, however when reporting the background or baseline characteristics of the participants, we feel it is not necessary. We hope that, after elaborating on our intention or aim with Table 1, we have clarified why the p value is not mentioned here. With regards to Table 2 we do not have a problem with reporting significance. We actually did report significance, however, at a different place within the manuscript: confidence intervals are reported in the main text as well as in Table 3 of our manuscript. Our intention was to keep Table 2 easy accessible and we aimed at easy interpretation of the table, reporting significance here would, in our opinion, decrease readability (as for instance both linear and logistics regressions are used). Therefore, we chose to report confidence intervals in the main text for all outcome measures, and we also reported the outcomes regarding the motivational factors (all tested with linear regression analyses with a non-inferiority approach) in Table 3. We hope this clarifies why we have made the current decision as well.

Reviewer 2

“The hypothesis is relevant and the methodology is aligned with the objectives. Qualitative data gathering with factor analysis for the conclusions derived would have strengthened the conclusions of the study. The study would be more meaningful if an attempt was made to identify causes of lesser credits in <7 group inspite of comparable intrinsic motivation. If the students in <7 group dropped out to meet academic requirements , it would establish the counter argument that intrinsic motivation alone is not enough for participation in ERP. Authors could highlight regarding remedial measures offered, if any for the students in <7 group to clarify its role in lesser in group difference in GPA3 as compared to GPA 1.”

We agree with the reviewer that a qualitative approach would be an interesting next step to further investigate the importance of motivation for research in selecting students for an extracurricular research programme (ERP), which is why we have added this as a future research avenue within the revised manuscript on page 22 (lines 474-475). We appreciate the interesting perspective mentioned by the reviewer to identify causes of lesser credits in the <7 group, and see this as a valuable direction for future research as well. Within our discussion section, we connected the lower credits to the drop-out rate. However, other factors might play a role as well which is why we have added this suggestion of the reviewer to our future research paragraph as well (see page 22, lines 471-472 of the revised manuscript). With regard to the reasons for drop-out, we indeed mentioned the possibility that students dropped out because of the academic requirements that need to be obtained in order to receive the certificate at the end of the extracurricular research programme. Every student can enrol within the ERP in our institute, the programme does not exclude students based on their first-year GPA. However, though this could be deemed as controversial, the institute does demand that students’ GPA for the regular medical educational programme is above 7 (on a 10-point grading scale) at the end of the ERP in order to receive the certificate. This not only sends out the implicit message that students scoring below 7 are not the type of students that are supposed to be enrolled within such ERPs (see manuscript page 18, lines 374-384), but it could also be demotivating and possibly decreases students initial intrinsic motivation for research. Especially when considered that this requirement, according to Self-Determination Theory, could well be an initiator of extrinsic motivation for research. However, it could also be that students indeed drop-out because of the academic requirement, but that they are still intrinsically motivated to conduct research as also mentioned on page 18 of the revised manuscript. With the current rules as they are, our findings could indeed be interpreted in such a way that intrinsic motivation alone is not enough to participate in the ERP, as also mentioned in our response to the first reviewer and now added within the revised manuscript on page 18 (lines 369-371). However, we feel that the academic requirement for a GPA of >7 at the end of the third year of medical training limits students that were initially intrinsically motivated for research to succeed within the programme and their journey towards a research oriented career. We hope that future research will shed light on actual reasons for drop-out, as it would be very interesting to gain in-depth knowledge on why some students persist within the programme while others drop-out, while both type of students might be the students that should be targeted for future physician-scientist careers. To conclude, no remedial measures were offered for the students in the <7 group.

Again, we would like to express our gratitude to you and the reviewers for the detailed and constructive comments. We are very thankful for the time invested in our manuscript. We really feel it helped us to improve our manuscript considerably, and hope you acknowledge our improvements.

If any questions or concerns remain, please do not hesitate to contact us. Thank you for receiving our revised manuscript, we look forward to our collaboration on this manuscript.

On behalf of all authors,

Kind regards,

Belinda Ommering

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

16 Aug 2021

PONE-D-21-11278R1

The importance of motivation in selecting undergraduate medical students for extracurricular research programmes

PLOS ONE

Dear Dr. Ommering,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Kindly revise as per the comments of the reviewer of this version of the manuscript to be considered further.

Please submit your revised manuscript by Sep 30 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Pathiyil Ravi Shankar

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The comments have been addressed and the project has to be taken forward as per suggestions to be more useful and relevant.

Reviewer #3: Reviewer 1 comments:

The importance of motivation in selecting undergraduate medical students for extracurricular research programmes (Manuscript Number PONE-D-21-11278R1)

This study attempts to throw light on the common oversight of an important benchmark in performance evaluation of medical researchers.

The paper is detailed, well explained and has incorporated good statistical tools. There a few observations made by the reviewer:

1. Kindly correct grammar inconsistencies at one or two areas in the paper (Eg. Inconsistencies with tenses: Line 273 – ‘All students were included’ is more appropriate as the study has been reported in the past tense throughout the paper)

2. If ERP drop-outs were more in the <7 group and this was statistically significant, one can argue that lack of motivation played an important role in the discontinuation from the ERP. Rather than dogmatically defending the findings with two contradictory theories for comparison of two different parameters between the same two study groups (viz. Reflected glory effect and Big fish in small pond theory), the acceptance to the paper can be enhanced with an open-mind approach that is:

- Authors can state in the paper that ‘the higher drop-out rate in <7 group could be justified by the big fish in small pond theory or could be attributed to the lack of motivation in the <7 group. Although the questionnaire shows no difference in motivation between the two groups, the finding of increased drop-outs confounds the result and warrants further research to investigate the reason for increased drop-outs in the <7 GPA group’

- If authors discuss that “Reflected glory effect” explains the increased self-perceived ability of the <7GPA group in the presence of >7 GPA group, authors should also mention that ‘from this perspective the presence of a >7 GPA group in the ERP is definitely a positive factor in the performance of <7 GPA group. However, the argument of the authors is not that GPA should not be a determinant factor of performance potential but the point here is that a GPA complemented with intrinsic motivation for research, research self-efficacy beliefs, perceptions of research and curiosity should be considered for selection of candidates for ERP in medical schools’.

If the authors accept the above-mentioned minor revision to the paper, the reviewer recommends that this paper be accepted for publication.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Reviewers comments.docx

Click here for additional data file.

3 Sep 2021

Dear Pathiyil Ravi Shankar,

We would like to thank you for the opportunity to revise our manuscript. In addition, we would like to thank the reviewers for the helpful review comments, which are greatly appreciated. The comments offer us guidance in the review process and help to improve our manuscript. Below we provide an overview on how we processed the review comments.

Journal requirements

“Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.”

Done as requested.

Reviewer 2

“The comments have been addressed and the project has to be taken forward as per suggestions to be more useful and relevant”

We would like to thank the reviewer for checking our revised manuscript and are pleased to read that the comments have been addressed.

Reviewer 3

“This study attempts to throw light on the common oversight of an important benchmark in performance evaluation of medical researchers. The paper is detailed, well explained and has incorporated good statistical tools.”

We would like to thank the reviewer for reviewing our manuscript and for his/her positive words.

1. Kindly correct grammar inconsistencies at one or two areas in the paper (Eg. Inconsistencies with tenses: Line 273 – ‘All students were included’ is more appropriate as the study has been reported in the past tense throughout the paper)

Done as requested (revised manuscript lines 273, 315, and 318).

“2. If ERP drop-outs were more in the <7 group and this was statistically significant, one can argue that lack of motivation played an important role in the discontinuation from the ERP. Rather than dogmatically defending the findings with two contradictory theories for comparison of two different parameters between the same two study groups (viz. Reflected glory effect and Big fish in small pond theory), the acceptance to the paper can be enhanced with an open-mind approach that is:

- Authors can state in the paper that ‘the higher drop-out rate in <7 group could be justified by the big fish in small pond theory or could be attributed to the lack of motivation in the <7 group. Although the questionnaire shows no difference in motivation between the two groups, the finding of increased drop-outs confounds the result and warrants further research to investigate the reason for increased drop-outs in the <7 GPA group’

- If authors discuss that “Reflected glory effect” explains the increased self-perceived ability of the <7GPA group in the presence of >7 GPA group, authors should also mention that ‘from this perspective the presence of a >7 GPA group in the ERP is definitely a positive factor in the performance of <7 GPA group. However, the argument of the authors is not that GPA should not be a determinant factor of performance potential but the point here is that a GPA complemented with intrinsic motivation for research, research self-efficacy beliefs, perceptions of research and curiosity should be considered for selection of candidates for ERP in medical schools’.”

The reviewer suggests that a lack of motivation played an important role in the discontinuation from the ERP and suggests that we should state in the paper that the higher drop-out rate could be justified by the big-fish-little-pond theory or could be attributed to the lack of motivation in the <7 group. We agree that the big-fish-little-pond theory is applicable here, which is why we mentioned this as a clarification in our discussion section. With regard to motivation, in general it could indeed be the case that motivation decreases and results in drop-out, however, we do not feel that it is justified to draw the conclusion from our data that a lack of motivation in the <7 group may cause the higher drop-out rate. When looking at the levels of intrinsic motivation at both timepoint 1 (table 1) and timepoint three (table 2) we can see that these levels are stable, or even marginally higher at timepoint three, implying that a lack of intrinsic motivation for research does not seem plausible. Especially as our non-inferiority approach shows that the <7 group is not inferior to the >7 group when it comes to intrinsic motivation for research. This view is corroborated by the coordinator of the ERP. We do believe that it could be that apparently motivation does not lead to finishing the programme, which we mention at page 18, line 364-366 of our revised manuscript (“it is remarkable that, though comparable in motivation for research, the drop-out in the <7 group is higher as compared to the ≥7 group, possibly implying that motivation does not lead to ERP completion”). Furthermore, the reviewer suggests that from the perspective of the reflective glory effect the presence of the ≥7 group is a positive factor for the performance of the <7 group, which we agree upon. Indeed, our argument is not that we should exclude the ≥7 group, rather that we should offer the <7 group the chance to participate within the ERP. In line with the reviewer’s suggestions, within the revised manuscript we now mention that ‘from this perspective, the presence of the ≥7 group is a positive factor for the performance of the <7 group’ (page 17, line 358-359).

Again, we would like to express our gratitude to the reviewers for the comments. We are very thankful for the time invested in our manuscript. We really feel it helped us to improve our manuscript considerably, and hope you acknowledge our improvements.

If any questions or concerns remain, please do not hesitate to contact us. Thank you for receiving our revised manuscript, we look forward to our collaboration on this manuscript.

On behalf of all authors,

Kind regards,

Belinda Ommering

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

11 Oct 2021

PONE-D-21-11278R2The importance of motivation in selecting undergraduate medical students for extracurricular research programmesPLOS ONE

Dear Dr. Ommering,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The reviewer has requested some minor revisions to your manuscript.

Please submit your revised manuscript by Nov 25 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Pathiyil Ravi Shankar

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

Reviewer #4: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: Reviewers concerns have been appropriately addressed; the manuscript may be accepted for publication in ite revised format.

Reviewer #4: The present manuscrit is very interesting.

Especially I strongly believe that lower academic curriculum should not be an "exclusion criteria" for further carrer.

Some issues.

Introduction is well written but I think it should be shortened to the focus of this paper, that is the impact of "selection" of students for research

Methods: due to low sample size normal distribution shold be assessed

Methods/results: I think that income of the famly of origin, and the level of education should be appraised as they impact of cv and opportunities

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

Reviewer #4: Yes: Fabrizio D'Ascenzo

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

3 Nov 2021

Dear Pathiyil Ravi Shankar,

We would like to thank you for the opportunity to revise our manuscript. In addition, we would like to thank the reviewers for the helpful review comments, which are greatly appreciated. The comments offer us guidance in the review process and help to improve our manuscript. Below we provide an overview on how we processed the review comments.

Journal requirements

“Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.”

Done as requested.

Reviewer 3

“Reviewers concerns have been appropriately addressed; the manuscript may be accepted for publication in the revised format”

We would like to thank the reviewer for checking our revised manuscript and are pleased to read that the reviewer suggests to accept the manuscript for publication.

Reviewer 4

“The present manuscript is very interesting. Especially I strongly believe that lower academic curriculum should not be an "exclusion criteria" for further career.”

We would like to thank the reviewer for reviewing our manuscript and for his positive words. We strongly agree with the reviewer that lower academic performance should not be an exclusion criteria.

“Some issues. Introduction is well written but I think it should be shortened to the focus of this paper, that is the impact of "selection" of students for research”

We agree with the reviewer that the focus of our paper is on selection of students for extracurricular research programmes (ERPs) and we are pleased to read that this came across as such. This study is situated in the context of a bigger problem that needs to be solved, i.e. the physician-scientist shortage, which is one of the reasons many ERPs were implemented. Therefore, we do feel that it is necessary to describe this problem and the reasons for implementing an ERP, logically following into the group that should be targeted to reach the desired outcome (i.e. more students pursuing a physician-scientist career). In this latter phase, questions regarding selection arise for which we investigated students with lower and higher academic performance within the ERP. We do feel, however, that we should keep the bigger picture in mind and provide this bigger picture to the broad readership as well. While still making sure that we did not lose the bigger picture and we did not go against previous reviewer requests to expand some parts within our introduction, we critically went through our manuscript and somewhat shortened the introduction at multiple points. We hope the editor understands our considerations and is satisfied with the current shortening of our introduction.

“Methods: due to low sample size normal distribution should be assessed”

We indeed assessed normal distribution at the start of our analysis phase. Data was normally distributed. For clarity purposes, we now mention that we established normal distribution of the data explicitly within the ‘analyses’ paragraph (page 11, line 242).

“Methods/results: I think that income of the family of origin, and the level of education should be appraised as they impact of cv and opportunities”

The reviewer suggests that income and level of education of the family of origin should be taken into account as they might impact cv and opportunities. We agree with the reviewer that these are factors that could certainly play a role in general performance, educational or career choices, and being selected for, for instance, an extracurricular research programme. However, we are afraid we do not have collected these data and are therefore not able to appraise them in the current study.

Again, we would like to express our gratitude to the reviewers for the comments. We are very thankful for the time invested in our manuscript. We really feel it helped us to improve our manuscript considerably, and hope you acknowledge our improvements.

If any questions or concerns remain, please do not hesitate to contact us. Thank you for receiving our revised manuscript, we look forward to our collaboration on this manuscript.

On behalf of all authors,

Kind regards,

Belinda Ommering

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

5 Nov 2021

The importance of motivation in selecting undergraduate medical students for extracurricular research programmes

PONE-D-21-11278R3

Dear Dr. Ommering,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Pathiyil Ravi Shankar

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: Yes: Fabrizio D'Ascenzo

10 Nov 2021

PONE-D-21-11278R3

The importance of motivation in selecting undergraduate medical students for extracurricular research programmes

Dear Dr. Ommering:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Pathiyil Ravi Shankar

Academic Editor

PLOS ONE