Verbal communication of students with high patient-physician interaction scores in a clinical performance examination assessed by standardized patients.

PURPOSE: Standardized patients (SPs) tend to rate medical students' communication skills subjectively and comprehensively, in contrast to such objective skill set defined in the clinical performance examination (CPX). Meanwhile, medical school instructors have a different approach in their evaluation of students' communication skills. We aim to analyze medical students' verbal communication skills using objective methods, and to determine the contributing factors of a patient-physician interaction (PPI) score.
METHODS: Students with high- and low-ranking scores for PPI in CPX were selected. The Roter interaction analysis system was used to compare verbal communication behaviors of the students and SPs. Patient-centeredness scores (PCSs), physician's verbal dominance, and number of utterances were compared between the two groups.
RESULTS: PCSs and physician's verbal dominance had no difference between the groups. The number of utterances during the limited time of 5 minutes of CPX was higher for the high-ranking students. They tended to employ more paraphrase/check for understanding, and closed questions for psychosocial state and open questions for medical condition. The SPs interviewed by high-ranking students gave more medical information and requested for more services.
CONCLUSION: In the case of the routine checkup, smooth conversations with more frequent utterances were detected in the high-ranking students. More medical information exchange and requests for services by SPs were higher for the high-ranking students. Medical communication instructors should keep in mind that our results could be indicators of a high PPI score.

Introduction

Communication skills have been made essential in medical education curricula because it is fundamental to facilitate rapport building and the efficient exchange of information during consultation [1]. A good patient–doctor relationship has an effect on optimal clinical outcomes [2].

Medical communication skills are evaluated as part of clinical performance, especially in the patient–physician interaction (PPI) section in the clinical performance examination (CPX). Standardized patients (SPs) evaluate the medical students based on their observations as soon as the student leaves the exam room. The professors and SPs showed no difference in their assessment of students’ attitudes, empathy and communication skills in CPX [3,4]. However, another study indicated that assessments may be vulnerable to errors, such as when SPs at the same CPX station have different physical presentations and facial expressions [5]. Consequently, the accuracy and reliability of SPs’ marks on the CPX checklist have been questioned for years. For items with clear evaluation criteria, such as history taking, SPs and professors tend to give mostly similar marks, but for those involving subjective judgment, such as PPI, they give significantly different scores [6]. Professors evaluate PPI as a direct or indirect observer, whereas SPs evaluate PPI after they have experienced consulting with a medical student. McLaughlin et al. [7] reported that SPs gave higher scores to students than physicians, and found a weak correlation between the SPs ’ and physicians’ scores. In addition, CPX scores by SPs were not correlated with multiple choice questions examination scores, unlike physician scores. Therefore, SPs may have better evaluated communication skill, whereas physicians have better evaluated problem-solving and clinical skills [7].

Well-trained and experienced SPs can evaluate PPI, and they can give feedback to medical students as replacement for professors. However, they are not the ones who teach or train medical students in the aspects of medical communication skill and clinical competency. It is the medical college professor, generally physicians who teach communication skills to medical students because SPs do not have knowledge of medical competency. The professors’ knowledge and understanding of the PPI may differ than those of the SPs who experience the PPI directly. Therefore, professors and SPs might have different standards for evaluation.

As mentioned, SPs tend to assess students subjectively and comprehensively in communication skills in CPX, whereas PPI is needed to facilitate analysis using objective and clear methods. The linguistic dialog analysis approach has been used to analyze PPI in CPX. The SPs and authors had differences in terms of PPI evaluation, but linguistic dialog analysis approach is considered a subjective tool [8]. In the present study, the verbal communication behaviors of medical students and SPs were analyzed and compared between a high- and a low-ranked group based on the PPI score. Analyses of medical encounters using objective tools can provide detailed and structured tips to students with a lower PPI score. Further, even high-ranking students may learn areas for improvement with respect to better communication.

Methods

1. Setting and samples

The course of doctor competency development runs for 4 consecutive weeks for first-year medical students at Inje University as of March 2015. It consists of modules for basic communication skills, basic physical examination, introduction to problem-based learning, self-development of competence, and clinical reasoning. The basic communication skills course consists of six topics: medical interview experience, agenda setting, attentive listening, empathy, medical interview with pediatric patients, and patient education. Each topic is covered in 4-hour class: (1) 1-hour lecture, (2) 2-hour exercise with SPs and formative feedback, and (3) 1-hour discussion and summary with the instructor. During the 2-hour exercise with SPs, six to seven students meet one SP. Each student takes turns in the practice medical interview with an SP for 5 minutes. Feedback by the SP and peer students is given to the student immediately after the interview.

Three stations of CPX are integrated into the course as summative assessment at the end of the course: physical examination for the abdomen, physical examination for the heart, and interview for routine checkup. Each student has three CPX stations. Three copies of three CPX were operating at the same time. Built-in video camera equipment and portable microphones were used for recording all three routine checkup CPX rooms. These recordings were originally made available for the remediation of students who fail to pass the communication CPX. Each student had only one video recording for the routine checkup CPX. As routine checkup is the only one to assess communication skills, this station was selected for the present study. Video recordings were used for our study after approval by the Institutional Review Board of Inje University Busan Paik Hospital (IRB number: 2015-08-0145).

The case was developed by the medical communication committee at the Inje University College of Medicine. As first-year students do not learn clinical medicine yet, a regular checkup case was developed. The case was of a 44-year-old female with a 5-minute interview for regular checkup. The SPs learned to present the social background as well as family and personal medical histories. They were also trained to provide specific answers to the expected specific questions of the medical student according to the scenario, and then rate the students’ history taking and PPI by SP trainer at Inje University College of Medicine. The instructions on what to do in a station were presented before the students enter the examination room. The student should build rapport, set the agenda, take medical history, family history, social history, and educate the patient. Three female SPs were recruited for the examination, and their real age was early fifties and have more than five experiences as SPs. One SP in each station met 32 to 33 medical students. History taking included four items, to be scored 0 or 4 (score of 0 to 16). PPI included five items, and scored from 0 to 4 (score of 0 to 20). The PPI rating scale used was the same marking sheet of the Busan–Gyeongnam consortium (Appendix 1) [9].

A total of 98 students participated in the CPX in March 2015. From this sample, 66 video recordings from two CPX rooms were available for analysis. The rest of the students were not included because of technical failure in the video recording from one CPX room. For this study, the study subjects were selected based solely on ranks of their PPI score. Ten students with high marks and another set of 10 students with low marks were selected at the author ’s discretion.

2. Roter interaction analysis system and coding

Roter interaction analysis system (RIAS) was chosen to code the verbal communication skill test in CPX. RIAS is a specific tool for coding medical dialogue between patients and providers, and the dynamics and consequences between them can be viewed [10]. It is widely used in medical exchange research for its reliability and validity. RIAS categories generally consist of task-focused behaviors and affective dimensions. Task behaviors include gathering data for understanding the patient’s problems as well as providing education and counseling to patients on illness and motivating patients to undergo the appropriate treatment. Affective behaviors include building relationships and patient–physician partnership [10]. Each utterance is assigned to one of 41 categories. An utterance was defined as a complete thought by either patient or physician, and ranges from a single word to a complete sentence. The 41 types of utterances can be classified under larger functional groupings: building a relationship, patient activating and partnering, data gathering (or question asking for patients), patient education, and counselling [10,11].

Coding is conducted directly from the audiotapes or videotapes without transcription [10]. In the present study, subjects’ videotaped CPX interviews were transcribed into text to avoid coding error, and then the two authors (KHP, HR) independently assigned each utterance to one of the 41 RIAS categories after watching videotaped recordings. Finally, one author (KHP), who attended a RIAS workshop, confirmed the types of each utterance, after discussion between two coders.

3. Study measures

Frequency (the number of utterances) of students’ communication behaviors were compared between the two groups.

Patient centeredness scores (PCS) were calculated based on the RIAS coding, and then compared between the two groups. PCS is defined in the Table 1. The difference between PCS 1 and PCS 2 is whether a physician’s biomedical information giving is treated as patient or doctor centered. Physician biomedical information giving is considered a biomedical detail in PCS 1, but as a patient-centered detail in PCS 2. Both scores are considered patient centered when they are over 1. PCS 1 has been the most commonly used in RIAS research, and published scores usually range from 0 to 5 [11,12].

Table 1.

Basic Characteristics, Patient Centeredness, and Physician’s Verbal Dominance between the Two Groups

Characteristic	High rank (n=11)	Low rank (n=10)	Total (n=66)
CPX score
History taking score (0-16)[**]	12.00±2.53	2.80±3.29	8.06±4.12
Physician–patient interaction score (0-20)[**]	12.00±0.89	7.30±0.82	9.55±1.81
Total score (0-36)[**]	24.00±2.53	10.10±3.25	17.61±4.92
Total no. of utterances
All student doctors	519	404
All standardized patients	376	273
Both students and standardized patients	895	676
Mean utterances per station
Student doctor[*]	47.18±5.71	40.40±9.22
Standardized patient[**]	34.18±4.31	27.20±3.46
Both student and standardized patient[**]	81.36±5.32	67.60±10.31
Patient centeredness score 1	1.16±0.57	1.59±1.08
Patient centeredness score 2	1.56±1.06	2.10±1.22
Physician’s verbal dominance	1.41±0.28	1.50±0.39

Data are presented as mean±standard deviation.

Patient centeredness scores 1=(all physician psychosocial data gathering, physician psychosocial information giving, physician emotional rapport building, physician engagement, patient psychosocial questions, patient psychosocial information giving, patient emotional rapport building, patient biomedical questions)/(all physician biomedical data gathering, physician biomedical information giving, physician procedural communication, patient biomedical information giving); Patient centeredness score 2=(all physician psychosocial data gathering, physician psychosocial information giving, physician biomedical information giving, physician emotional rapport building, physician engagement, patient psychosocial questions, patient psychosocial information giving, patient emotional rapport building, patient biomedical questions)/(all physician biomedical data gathering, physician procedural communication, patient biomedical information giving).

CPX: Clinical performance examination.

p<0.05,

p<0.01,

Mann-Whitney U-test.

Physician’s verbal dominance, or the ratio of the sum of physician statements divided by the sum of patient statements, was also calculated [13].

4. Statistical analysis

Continuous variables between the two groups were analyzed with Mann-Whitney U-test: PCS 1, PCS 2, physician’s verbal dominance, and frequencies of students’ communication behaviors. All continuous variables were then described as mean and standard deviation. The IBM SPSS ver. 20.0 (IBM Corp., Armonk, USA) was used for the analyses, and significance was declared at p<0.05.

Results

1. Student characteristics and basic results

Table 1 showed that the mean of the history taking score and PPI score in the total of 66 students were 8.06±4.12 and 9.55±1.81, respectively. Based on PPI scores, 11 high-ranking students (two students scored tie) and 10 low-ranking students were selected. The mean PPI score was 12.00 for the high-ranking students and 7.30 for the low-ranking ones. The former group also scored higher in history taking.

A total of 1,571 utterances were detected. For the high-ranking students, 895 utterances were detected, with the average number of utterances of 81.36 per exam. For the low-ranking students, 676 utterances were detected, with an average of 67.60 per exam. The mean of the frequency of utterances in all students, all SPs, and both groups were significantly higher for the high-ranking students.

PCS 1 and PCS 2 had no differences between the two groups. The same trend was seen for physician ’s verbal dominance (Table 1).

2. Patterns of utterances

Interrater agreement was high (κ=0.935, p<0.001). The most common type of utterance for high-ranking students was a paraphrase/checks for understanding, whereas that for low-ranking students was closed questions for the medical condition (Table 2). Medical information giving is the most common utterance of SPs in both group (Table 3).

Table 2.

Comparison of Communication Frequencies of Student Doctors: Type of Utterances

Type of utterances	High rank (n=11)	Low rank (n=10)
Social talk
Personal remarks, social conversation	5.19±0.98	4.70±1.34
Positive talk
Laughs, tells jokes	0.09±0.30	0.00±0.00
Shows approval (direct)	0.55±0.69	0.50±1.08
Gives compliment (general)	0.18±0.40	0.00±0.00
Shows agreement or understanding	0.55±0.82	0.80±1.55
Negative talk
Shows disapproval (direct)	0.00±0.00	0.00±0.00
Shows criticism (general)	0.00±0.00	0.00±0.00
Emotional talk
Empathy statements	1.27±1.19	1.60±0.84
Legitimizing statements[*]	0.64±0.81	0.00±0.00
Partnership statements (physician only)	0.09±0.30	0.00±0.00
Self-disclosure statements (physician only)	0.36±0.92	0.00±0.00
Shows concern or worry	0.45±0.69	0.80±0.79
Reassures, encourages or show optimism	0.64±0.92	0.70±1.34
Participatory facilitators
Back-channel responses (physician only)	0.00±0.00	0.00±0.00
Asks for reassurance	0.00±0.00	0.00±0.00
Asks for opinion (physician only)	0.00±0.00	0.10±0.32
Asks for permission (physician only)	0.73±0.79	0.20±0.42
Paraphrase, checks for understanding	7.64±3.70	4.90±2.81
Asks for understanding	0.00±0.00	0.00±0.00
Procedural talk
Transition words	0.09±0.30	0.10±0.32
Gives orientation, instructions	4.18±2.48	2.70±2.91
Unintelligible utterances	0.09±0.30	0.00±0.00
Closed questions
Medical condition	6.27±2.83	7.00±2.87
Therapeutic regimen	6.27±2.83	7.00±2.87
Lifestyle	3.18±2.09	4.10±2.42
Psychosocial[*]	0.73±0.65	0.20±0.42
Other	0.73±0.65	0.20±0.42
Open questions
Medical condition[*]	4.09±2.07	2.20±1.32
Therapeutic regimen	0.00±0.00	0.10±0.32
Lifestyle	0.64±0.67	1.40±1.17
Psychosocial	0.00±0.00	0.10±0.32
Other	0.27±0.47	0.20±0.42
Bid for repetition	0.00±0.00	0.00±0.00
Gives information
Medical condition	3.73±2.53	4.50±2.80
Therapeutic regimen	3.73±2.53	4.50±2.80
Lifestyle	0.91±1.14	1.00±1.05
Psychosocial	0.55±1.04	0.00±0.00
Other	0.00±0.00	0.00±0.00
Counsels or directs behavior (physician only)
Medical condition/therapeutic regimen	0.91±1.70	0.70±1.06
Lifestyle/psychosocial	1.91±3.21	1.30±1.70
Total[*]	47.18±5.71	40.40±9.22

Data are presented as mean±standard deviation.

p<0.05,

Mann-Whitney U-test.

Table 3.

Comparison of Communication Frequencies of Standardized Patients: Type of Utterances

Type of utterances	High rank (n=11)	Low rank (n=10)
Social talk
Personal remarks, social conversation	1.73±0.90	1.80±1.40
Positive talk
Laughs, tells jokes	0.09±0.30	0.00±0.00
Shows approval (direct)	0.00±0.00	0.10±0.32
Gives compliment (general)	0.00±0.00	0.00±0.00
Shows agreement or understanding	4.36±2.42	3.90±2.13
Negative talk
Shows disapproval (direct)	0.45±0.69	2.20±2.44
Shows criticism (general)	0.00±0.00	0.00±0.00
Emotional talk
Empathy statements	0.00±0.00	0.00±0.00
Legitimizing statements	0.00±0.00	0.00±0.00
Shows concern or worry	0.55±0.69	0.30±0.48
Reassures, encourages or show optimism	0.36±0.50	0.10±0.32
Participatory facilitators
Asks for reassurance	0.09±0.30	0.00±0.00
Paraphrase, checks for understanding	0.27±0.65	0.00±0.00
Asks for understanding	0.00±0.00	0.00±0.00
Requests for services (patient only)[*]	0.64±1.03	0.00±0.00
Procedural talk
Transition words	0.00±0.00	0.00±0.00
Gives orientation, instructions	0.00±0.00	0.00±0.00
Unintelligible utterances	0.00±0.00	0.00±0.00
All questions (patients)
Medical condition	0.09±0.30	0.10±0.32
Therapeutic regimen	0.00±0.00	0.00±0.00
Lifestyle	0.00±0.00	0.00±0.00
Psychosocial	0.09±0.30	0.00±0.00
Other	0.00±0.00	0.00±0.00
Bid for repetition	0.00±0.00	0.10±0.32
Gives information
Medical condition[**]	18.27±4.98	11.00±2.62
Therapeutic regimen	0.18±0.40	0.10±0.32
Lifestyle	6.27±2.57	6.80±3.43
Psychosocial	0.55±1.21	2.40±7.24
Other	0.09±0.30	0.10±0.32
Total[**]	34.18±4.31	27.30±3.40

Data are presented as mean±standard deviation.

p<0.05,

p<0.01,

Mann-Whitney U-test.

In terms of type of utterance, the high-ranking students used more paraphrase/checks for understanding, legitimizing statements, closed questions about psychosocial state and open questions about medical information compared with the low-ranking students. SPs who had consulted with high-ranking students requested for more services and more freely gave information on their medical condition compared with the low-ranking group (Table 3). Requests for services by SPs were detected in the four samples of high-ranking students group. An example was that the SP said, “Please perform a checkup of my body, and examine other areas that may need attention.”, when the student asked the SP her health problems and negotiated the agenda. In addition, the frequency with which both student groups used closed questions for medical information was similar, but high-ranking students used open questions more often. Accordingly, the SPs gave more medical information to high-ranking students.

Discussion

In this study, high-ranking students were determined by the PPI score evaluated by SPs. We consequently analyzed communication success from the SPs ’ viewpoint.

The frequency of utterance during the limited time of 5 minutes was higher in high-ranking students and their SPs, which indicates that high-ranking students had a smooth conversation with their SPs. In other words, they conversed with their SPs without any unnecessary pause, and more contents were exchanged. These results are similar to the previous study. Shin et al. [14] reported that female physicians had patient-centered encounters because they had more dialog with SP.

In medical practice, relationship building begins with personal remarks and social conversation. As all students initiate their medical encounters with personal remarks and social conversation, no difference was observed in the relationship building performance of both groups. Students might be nervous and stressed during encounters, based on the occurrence of few negative and positive talk. Emotional talk was also rare in both groups, but high-ranking students used legitimizing statements more often. Legitimizing statements indicated that it is normal for or understandable why SPs feel or act a certain way, and is a relatively easy form of emotional talk.

High-ranking, compared with low-ranking, students used paraphrasing/checks for understanding more often, a technique for participatory facilitation. Paraphrasing or clarification is an easy way to demonstrate attentive listening to SPs, while checks for understanding are a form of shared understanding. In RIAS coding, back-channel responses refer to the typical method of responding and listening attentively to a patient, encouraging them to talk. Interestingly, back-channel responses rarely occur in the Korean language; therefore, paraphrasing is used like the back-channel response in English. SPs might feel supported and encouraged to talk by students’ paraphrases. Since PPI is determined by the SPs’ subjective and comprehensive assessment, PPI can be correlated to their satisfaction about encounters with students. In the study conducted in the emergency department, patient satisfaction with a consultation is reportedly associated with the amount of talk relating to relationship building, patient activating, and partnering [15].

High-ranking students used more closed questions about psychosocial state and open questions about medical condition, when compared with their low-ranking counterparts. There was no difference in the frequency of closed questions about medical condition. Nevertheless, the SPs in high-ranking group showed the higher tendency to provide more medical information when responding to more open questions, leading to more freely shared medical information. This result might be attributed to the rapport between student and SP in the high-ranking group.

SPs who had consulted with high-ranking students requested for services more, such as those related to their concerns on their condition, when the student negotiated the agenda. Thus, the student should thoroughly explore the problems brought up by the SP.

In the current study, PCSs were not different between the two groups. The mean PCS 1 and 2 in both groups were greater than 1.0. As PCS is considered patient centered when greater than 1.0, both groups’ verbal communication can be considered patient centered. A reason is that, as aforementioned, the students and SPs in both groups had fewer emotional rapport-building opportunities. Meanwhile, when a physician’s biomedical information giving is included in the patient-centered side for PCS 2, PCS increased. Educating and counseling related to therapeutic regimens led to higher patient satisfaction [15]. Thus, biomedical information giving is regarded as a crucial patient-centered behavior.

In a research on actual patients, patient satisfaction was correlated with rapport building and psychosocial communication [16]. The factors contributing to patient satisfaction are various and dependent on the type of situation. For example, emergency patients were more satisfied with their consultations with emergency nurse practitioners than doctors, because emergency nurse practitioners provided more education and counseling on therapeutic regimens and patients could understand their condition and participate in the decision making [15].

Awkward laughs or pauses were noticed in the recordings of a number of students, such as when they could not find anything to say during the interview. As these utterances are part of paraverbal communication, they cannot be analyzed by RIAS. In addition, these behaviors might be a part of the reason that the count of utterances in the low-ranking students is low. Words themselves comprise only 7% of communication [17]. Therefore, nonverbal and paraverbal communication also need to be analyzed.

The present study has several limitations. First, the sample size is small. The authors selected only 11 high-ranking students and 10 low-ranking students to maximize the difference between the groups. In addition, the analysis was limited to only one CPX, routine checkup. Second, one author selected study subjects, and then conducted coding without being blind to the ranks of the students. The other authors conducted coding fully blinded. Although not being blind to the subjects could lead to an unfair result, interrater agreement was high. Third, nonverbal behaviors cannot be evaluated by RIAS. A description of nonverbal behaviors by SPs is the weakest category for medical students [18]. In addition, RIAS cannot measure all the potential domains of patient-centered communication [12]. RIAS categories are not equivalent to those in the PPI checklist. The RIAS can be used only for verbal communication behaviors, but not the propriety of the communication and sequence of utterances [19].

The number of utterances during the limited time of CPX was higher for the high-ranking students, which indicates that the process of conversation was active and smooth. Provision of more medical information (by SPs responding to open questions about medical conditions from students) could lead to high PPI scores during routine checkups. It may be the same when high-ranking students use more legitimizing statements and closed questions for psychosocial states and SPs request a service. Furthermore, nonverbal behaviors are considerable factors in medical communication, and therefore, nonverbal behaviors need to be analyzed.

No.	Patient-physician interaction score	Very good	Good	Average	Poor	Very poor
1	Formed decent bond and started the interview	4	3	2	1	0
	- Introduction, interest, respect, confidence & reliability
2	Listened closely	4	3	2	1	0
	- Open question, waiting, responding, listening attitude
3	Asked questions efficiently	4	3	2	1	0
	- Easy to understand, clarify, summarize, set up agenda
	- Avoided: misleading, multiple questions
4	Showed empathy to my concerns and emotions	4	3	2	1	0
	- Attitude, eye contact, empathy, understanding (patient concerns and thoughts)
5	Explained easily to understand	4	3	2	1	0
	- Knowledge and explanations, concise, easy vocabulary, questions
	- Avoided: impatient, unnecessary information