Is there an association between nurse, clinical teacher and peer feedback for trainee doctors' medical specialty choice? An observational study in Taiwan.

OBJECTIVES: This study explored whether there is an association between medical trainees' future specialty choices and the 360-degree feedback they receive. We hypothesised that the higher the scores that teachers, trainees and/or nurses give to postgraduate year 1s (PGY1s) in any given specialty, the more likely that they will choose that specialty for their residency.
SETTING: The study was conducted in a large regional teaching hospital in Taiwan. PARTICIPANTS: The participants of this study were n=66 PGY1s who had completed their medical studies domestically or internationally and had received their PGY1 training in a single teaching hospital in southern Taiwan. Data from 990 assessments were included. PRIMARY AND SECONDARY OUTCOME MEASURES: Logistic regression analyses for teachers', nursing staff and peers' authentic assessments of trainees were undertaken for (1) desired specialty, (2) applied specialty, (3) enrolled specialty, (4) consistency between desired and applied specialties, (5) consistency between applied and enrolled specialties and (6) consistency between desired and enrolled specialties. Alpha was set at p<0.05.
RESULTS: Nursing staff scores were significantly associated with all six dependent variables. Furthermore, teachers' scores were significantly associated with trainees' desired specialty and the consistency between desired and enrolled specialty. Peers' scores were not significantly associated with any dependent variable.
CONCLUSIONS: Trainees' specialty choices are associated with scores given by nursing staff and clinical teachers. We suggest that qualitative research methods should further explore this association to ascertain whether PGY1s are consciously influenced by these scores and if so, in what way. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This novel study overcomes problems with self-report data by mapping authentic assessments to trainees’ desired specialties, applied specialties and enrolled specialties.

This is the first study to examine the association between 360-degree feedback provided by nurses, doctors and peers and medical trainees’ career choices.

Participants in our study came from a single teaching hospital in one country.

The associations found need further exploration before any causal claims can be made.

Introduction

Understanding individuals’ motivations underpinning medical specialty choice is of importance to the development of a trainee’s career and has implications for the issue of workforce planning around future healthcare service provision.1 Workforce planning is problematic internationally: it requires careful consideration of how patients’ healthcare requirements are continually changing, alongside careful understanding of how medical student and trainee career intentions, preferences and destinations fluctuate over time.2–4 It is therefore of increasing importance that we understand the factors that might affect trainees’ intentions and preferences concerning career choices.

Specialty choice has been attributed to individual differences in personality or background demographics, including family, gender, age and marital status.5–8 Furthermore, these demographics interact with future expectations. These expectations might include the perceived benefits and attractiveness of specialties in terms of work–life balance, financial rewards, decisions about having a family, perceived personal skill set compatibility and intellectual rewards.6–11 As such, it has been argued that specialty choice, for some, is an unproblematic issue, as individuals’ personal characteristics and desires enable them to make a clear decision around where they feel they ‘fit’; thus, around half of medical students with a career preference prior to beginning their studies maintain their intentions throughout their education.12

For some, career choice can be difficult. Students can be influenced by a range of external factors, including conversations with friends and family, medical school curricula, medical school resources (indirectly), pressures produced from different specialties, positive role models, peer encouragement, influences on mental and physical health, learning effectiveness in different specialties, training settings and hidden curriculum issues (eg, negative comments about certain specialties and negative role models).8 12–23 However, timing of curricular interventions, specialty faculty-to-student ratios and student debt level have been found to have no impact on student career choice.8

Theoretical perspective

We draw on both sociological and psychological theories to understand the influences on students’ and trainees’ career choices. The sociologist, Cooley, proposed the theory of the looking-glass self.24 He pointed out that self-assessment is based on a person’s personal contact with others and their imagination of others’ judgements and assessments of them. This self-assessment comprises the following three main points of self-assessment based on (1) others’ attitudes towards them in society, (2) self-comparison with people who have similar social conditions and statuses and (3) their analysis of their psychological activities.

Based on this theoretical perspective, we propose the following hypothesis: high feedback scores to post-graduate year 1 (PGY1) trainees from teachers, peers and/or nurses in a particular specialty will be associated with PGY1s’ subsequent specialty choices for their medical residencies.

Medical career research in Taiwan

In response to the uneven distribution of medical personnel in Taiwan over the past few years, there has been an increase in the number of studies related to medical specialty choice.25–27 Most studies have conducted their analyses mainly on the subjects of market demand, gender, government policy, personal traits and personal preferences or interests. Frequently, questionnaires have been used to explore possible influencing factors. Many studies have demonstrated that personal preference, sense of achievement, learning experience, others’ expectations, promotion and financial reward and medical conflict are likely to influence medical students’ specialty choices, with the first three factors accounting for the highest percentages.28–32 Thus, when deciding on a career, individuals take both their personal learning experiences and environmental characteristics into consideration.33 However, these studies have been based on self-report data only.

360-degree assessment and career choice

Although previous studies have reported that curricular interventions have no impact on students’ career choices, it has been argued that trainees’ personal career choices can be triggered by the external impact of others’ assessments of them (as an environmental stimulant).34 Further, feedback from evaluators with a variety of viewpoints, such as that which is provided through 360-degree assessment (see online supplementary file), has the potential to influence trainees’ internal thinking and cognition, thereby increasing (or decreasing) their motives for choosing certain specialties.35 36

The postgraduate year is the training that all junior doctors complete following graduation. Experiences during this formative time can influence their medical specialty choice. By analysing authentic feedback from PGY1s’ teachers, nursing staff and peers, we aim to fill the gap in the literature by exploring the relative association between different evaluators’ assessments of trainees and their subsequent career choices.37 In doing so, we aim to address the following research question: are teachers’, peers’ and/or nurses’ assessments associated with PGY1s’ specialty choices?

Method

This study used authentic assessments from a 360-degree feedback assessment through which teachers, nursing staff and peers assess PGY1 trainees’ learning each month. Assessment items include medical knowledge, skills and learning attitude. The results of the feedback were sent to the PGY1s, teachers and supervisors to inform them about training effectiveness. This section will explain the study context, participants, research tool, procedures, data processing and analysis. This study was approved by the Institutional Review Board of the Chang Gung Memorial Hospital (104-6122B).

Setting

The study setting was a large teaching hospital in Taiwan. Taiwanese medical students undergo 7 years of training before they graduate from medical school. After graduation, they take the national license examination, at which point they undergo their postgraduate training for 1 year. Following this, PGY1s apply for their specialty of choice. They obtain their residency following one further year of training in their specialty of choice, at which point they are employed by the hospital. Thus, in Taiwan, training to become a doctor requires 8 years. For the PGY1s, the first year following graduation is important, as it represents a turning point in their training phase. During this time, they need to adapt to several changes in their environment, including working in various departments and interacting with different medical teams. This results in a myriad of challenges for the PGYs’ capabilities and adaptabilities.

In terms of the PGYs’ experiences, rotation training in Taiwan is mandatory in five major specialties (ie, internal medicine, surgery, obstetrics and gynaecology, paediatrics and emergency). To obtain comprehensive data from the assessments, we thus focused on these five specialties. Furthermore, since each department provides their training based on these five major departments, we categorised other departments under them. For example, the anaesthesia department was categorised under the surgery department, and the haematological oncology department was categorised under the internal medicine department.

Patient and public involvement

This study was to collect data from PGYs’ evaluation training of clinical teachers, peers and nursing staff during the training period. The study did not collect any relevant information from patients.

Participants

This study adopted a purposive sampling method.

PGY1s

The Ministry of Health and Welfare limits the yearly capacity of PGY1s in any single hospital to between 20 and 27. The participants in this study comprised n=66 PGY1s (78.8% men, similar to the gender ratio of Taiwan’s medical student population). The participants had completed their medical studies nationally or internationally. They had received and completed their PGY1 training programme in a single teaching hospital based in southern Taiwan between 2012 and 2014. Most participants were from three different Taiwanese universities, although n=5 were from overseas medical schools (7.6%). Each PGY1 rotated among five major specialties (ie, internal medicine, surgery, obstetrics and gynaecology, paediatrics and emergency medicine), with different evaluators (ie, teachers, nursing staff and peers) providing the scores. The response rate was 100%, since PGY1 training and 360-degree multisource feedback are compulsory in the Taiwanese system. Every trainee received at least one assessment from teacher, nurse and peer evaluators. That is, each PGY1 received 15 assessment forms from each of the three evaluators of the five specialties. The samples were gathered only after the scores from the three evaluator assessments were averaged. A total of 990 assessments (ie, 330 assessments from teachers, nurses and peers each) were collected (see figure 1).

Figure 1

Overview of the 360-degree assessments model and the time points for data collection. PGY1, postgraduate year 1.

Teachers

Each department had one attending physician assuming the role of evaluator. In total, 127 teachers participated in the assessment. When more than one teacher provided an assessment to the same PGY1, the PGY1’s score in that division became the average of the scores given by all teachers. The teachers, as teachers of primary care medicine, must be equipped with the skills for teaching the six core competencies set by the Accreditation Council for Graduate Medical Education. They also must complete a minimum of 40 hours of training to ensure their quality as a teacher prior to assessing the trainees’ performance.

Nursing staff

As above, each department had one senior nurse acting as the evaluator. Head nurses from different wards or the designated senior nursing staff (working for more than 3 years) were responsible for assessing the PGY1s. Consensus meetings were held prior to the actual scoring to ensure standardised assessment. In total, 116 nursing staff participated in the assessment. When more than one nurse gave an assessment to the same PGY1, the PGY1’s score in that division comprised the average of the scores provided by the nurses.

Peers

Peers comprise the PGY1s who are in the same rotation as the assessed PGY1 or the chief resident. The PGY1s have much assessment experience. In total, 143 peers participated in the assessment. When more than one peer provided a score to the PGY1, the PGY1’s score in that division comprised the average of the scores provided by their peers.

Procedure

The assessment was undertaken during the PGY1s’ training period. After completing their rotation in each division, the assessors used the 360-degree feedback form to provide their assessment. The results were collected and organised by the administrative staff who returned the feedback to the PGY1s. After the training was completed, the data for the PGY1s’ desired specialties, applied specialties and enrolled specialties were collected separately.

Data processing and analysis

The valid data were entered into SPSS V.20, proofread and statistically analysed. As this study draws on data from different years, some evaluators had different scoring criteria. We therefore used the following procedures: we entered the scores from each group (ie, teachers, nurses and peers) in the same department into SPSS to calculate the standard score (z score). This revealed the SD between the original score and the mean, which was compared afterwards. A panel of two or three experts shared their understanding of the criteria for scoring prior to the assessment. The consistency of the evaluators was checked to confirm the scoring reliability using descriptive statistics and logistic regression analysis.

We undertook logistic regression analyses to examine each dependent variable (desired, applied and enrolled specialties) against each of the assessment scores (teachers, nurses and peers). An alpha level of p<0.05 was set. The categorical dependent variables were divided into two types: specialty selected and specialty not selected. Thus, in the dependent variables, when the medical interns enrolled in the PGY1, their desired specialties during their residency were obtained. After finishing their PGY training programme, they then applied for their specialty for their residency. Finally, they were enrolled in a specialty. This three-stage process leads to discrepancy in ‘specialty choice.’ For example, in the five major specialties (ie, internal medicine, surgery, obstetrics and gynaecology, paediatrics and emergency), if the PGY1’s desired specialty was the same as the enrolled specialty, then a 1 would be assigned; otherwise a 0 would be assigned. The discrepancies were analysed. Scores given by the evaluators (ie, the teachers, nursing staff and peers) across the different specialties were analysed as independent variables. Apart from the p value, which indicates the level of significance, another important aspect of analysis was the OR. When the OR was greater than 1, the more probable it was that the PGY1 trainee would choose that specialty.

Results

Teachers’ assessment and specialty choice

The scores given by the teachers (independent variable) were analysed against the six specialty choices. Table 1 displays the data for the dependent variables.

Table 1

Logistic regression of teachers’ assessment against each dependent variable

Dependent variable	B	SE	Wals	df	P values	OR	95.0% CI for OR
							Lower	Upper
Desired specialty	0.396	0.186	4.505	1	0.034	1.486	1.031	2.141
Applied specialty	0.355	0.193	3.379	1	0.066	1.427	0.977	2.084
Enrolled specialty	0.192	0.193	0.995	1	0.319	1.212	0.831	1.768
Desired specialty-enrolled specialty consistency	0.351	0.225	2.435	1	0.119	1.420	0.914	2.206
Applied specialty-enrolled specialty consistency	0.358	0.216	2.742	1	0.098	1.431	0.936	2.187
Desired specialty-applied specialty consistency	0.348	0.174	4.014	1	0.045	1.416	1.008	1.990

A significant association was found between the desired specialty and teachers’ assessment (p=0.034) and the desired specialty-applied specialty consistency (p=0.045). However, the association between teachers’ assessment and PGY1s’ applied specialty did not reach a level of significance.

Nurses’ assessment and specialty choice

For nurses, all six dependent variables reached a level of significance: the desired specialty (p=0.018), the applied specialty (p=0.029), the enrolled specialty (p=0.006), the desired specialty-enrolled specialty consistency (p=0.018), the applied specialty-enrolled specialty consistency (p=0.029) and the desired specialty-applied specialty consistency (p=0.015). Table 2 displays the data for the logistic regression of nurses’ assessments.

Table 2

Logistic regression of nursing staff assessment against each dependent variable

Dependent variable	B	SE	Wals	df	P values	OR	95.0% CI for OR
							Lower	Upper
Desired specialty	0.415	0.176	5.554	1	0.018	1.515	1.072	2.140
Applied specialty	0.400	0.183	4.783	1	0.029	1.492	1.042	2.136
Enrolled specialty	0.513	0.187	7.525	1	0.006	1.670	1.158	2.409
Desired specialty-enrolled specialty consistency	0.507	0.215	5.581	1	0.018	1.660	1.090	2.528
Applied specialty-enrolled specialty consistency	0.449	0.205	4.775	1	0.029	1.566	1.047	2.342
Desired specialty-applied specialty consistency	0.401	0.165	5.900	1	0.015	1.493	1.080	2.063

Peers’ assessment and specialty choice

For peers, none of the dependent variables were significantly associated with specialty choice. table 3 displays the data for the logistic regression of peers’ assessments.

Table 3

Logistic regression of peers’ assessments against each dependent variable

Dependent variable	B	SE	Wals	df	P values	OR	95.0% CI for OR
							Lower	Lower
Desired specialty	−0.012	0.171	0.005	1	0.944	0.988	0.706	1.382
Applied specialty	−0.085	0.178	0.228	1	0.633	0.919	0.648	1.302
Enrolled specialty	0.029	0.181	0.026	1	0.872	1.029	0.722	1.467
Desired specialty-enrolled specialty consistency	0.021	0.209	0.010	1	0.921	1.021	0.678	1.536
Applied specialty-enrolled specialty consistency	0.092	0.202	0.210	1	0.647	1.097	0.738	1.630
Desired specialty-applied specialty consistency	0.031	0.161	0.038	1	0.846	1.032	0.753	1.413

Discussion

We undertook a series of analyses to examine the associations between teacher, nurse and peer feedback and PGY1s’ specialty choices. We identified a high correlation between nursing staff assessment and PGY1 specialty choice. This finding is novel and, to our knowledge, is the first time that such an association has been demonstrated. Although we are unable to ascertain any causal attributes, a positive relationship between nurses and PGY trainees has been previously reported. For example, research has identified the pivotal role of nursing staff on junior doctors’ first year of practice, both in terms of practical and emotional support.38–40 In terms emotional support, the first year of medical practice is notoriously stressful, as newly graduating doctors struggle with their new roles and responsibilities.41 Compared with clinical teachers, nursing staff are often seen as being more caring towards PGY1s.42 43 It is therefore unsurprising that junior doctors look to them for emotional support in times of crisis.38 Further, nurses tend to be more stable in terms of work placements (ie, not rotating across specialties, as with doctors in training). In other words, compared with the PGY1s’ peers, nurses are likely to have a better understanding of the department in which they are working. In addition, compared with clinical teachers, nurses are likely to spend more time with the PGY1s. As a result, nurses might understand the PGY1s better than their peers and clinical teachers in terms of their training and support requirements. As such, they are able to provide them with practical advice.38–40 It might be that nurses’ positive feedback, alongside the other support provided by them during placements, could be a contributing factor in junior doctors’ desire to work in that specific area due to the overall perceived support they might feel and the relative stability that nurses contribute to the workplace. Therefore, the association we have found merits further investigation to ascertain the degree to which this has a causative element.

Second, we found a significant association between teachers’ scores and both PGY1s’ desired and desired-applied specialty scores. Again, while being careful not to attribute this as a causative association, this finding does resonate with the plethora of research that has been undertaken in medical education around the influence of role models on medical students and junior doctors.21–23 Thus, it has long been recognised that aspects such as role models’ personality, enthusiasm, approachability, clinical skills, competence and teaching ability all influence medical students’ career choices. Adding teachers’ positive assessment of trainees to the list of influencing factors also makes sense theoretically. Consider the concept of Cooley’s looking-glass self.24 When we perceive ourselves through others’ attitudes towards us, especially the attitudes of others to whom we feel an affinity or admiration, it is possible that their positive assessment might influence our future hypothetical identity: the ‘who I might be.’ Once again, the association we have found merits further exploration to ascertain the exact relationship between assessment and career choice, and whether the ‘who I might be’ is reinforced by such positive feedback.

That we found no significant association between peers’ positive evaluations of PGY1s and their subsequent specialty choices might at first seem at odds with previous research that has suggested that peer encouragement plays a role.23 However, on closer inspection, it appears that the effect of peers on medical specialty choice has not been found in a PGY1 cohort, and findings around this peer effect on specialty choice are mixed, with some studies suggesting the influence of peers to be minimal at best.44 Despite this, the inclusion of an exploration of the relative impact of peers on PGY1s’ career choices would be a useful addition to any future study in this area.

As with any study, our research has limitations. First, we have only examined the association between assessment and career choice and therefore make no claim regarding causality. Although a necessary first step, as we have highlighted, further research needs to be undertaken before this link can be established. We suggest that qualitative research methods should explore this association in the first instance to ascertain whether PGY1s are consciously influenced by these scores and if so, in what way. Second, participants in our study come from a single teaching hospital in southern Taiwan. Furthermore, although the gender ratio of our participants reflects that of trainees in Taiwan, it should be noted that gender ratios differ dramatically around the world. As such, the extent to which our findings can be generalised to account for other trainees’ career choice motivations should take this into consideration. Third, we have only considered the association of assessment on specialty choice. Other aspects such as lifestyle, the time medical staff spend on call, and trainees’ perceptions of which specialties are ‘popular’ are also associated with career choice. Furthermore, as previously mentioned, by the time they reach their PGY1 year, many trainees have already made their decisions regarding specialty choice. In turn, positive assessments of trainees might be influenced by their specialty choice: when trainees are working in the specialty of their choice, they might demonstrate a greater aptitude for their work or have a more positive attitude, which leads them to receive more positive assessments. This might be more so for nurse assessors who have less training than the clinical educators in terms of trainees’ assessment. Finally, during their PGY training programme, PGY1s also face many other workplace-based assessments of clinical competence. Thus, the mutual influences among all these assessments could influence their specialty choices.

Despite these limitations, our study makes a valuable contribution to the literature, in that it is a novel study examining the respective association between teachers’, nurses’ and peers’ feedback and PGY1s’ specialty choices.

In this study, we examined the association between PGY1s’ specialty choices and certain aspects of their clinical learning experiences. However, a major part of their learning experience comprises interactions with patients and their families. Furthermore, patients have also been included as part of the 360-degree appraisal of doctors in some contexts.45 Although medical students’ interactions with patients has been demonstrated as being influential in their career choice of surgery,46 to date, there has been no research to our knowledge examining whether feedback from patients and their families is associated with PGY1s’ career specialty choices. Future research could examine this factor alongside an in-depth, qualitative examination of trainees’ specialty choices. A deeper understanding of the factors that affect trainees’ specialty choices is required to facilitate ongoing workforce planning activities.