Evaluation and improvement of doctor-patient communication competence for emergency neurosurgeons: a standardized family model.

Disease treatments have been significantly influenced by the communications between patients, their families, and doctors the lack of which may lead to malpractice allegations and complaints. In particular, inadequate communication may delay diagnosis and treatment. Therefore, for doctors communication and interpersonal skills, are as important as clinical skills and medical knowledge. In this study we intended to develop two detailed communication content checklists and a modified interpersonal skills inventory, aiming to evaluate their integrity in the midst of communication skills assessments, to provide feedback for some participants, and to observe their communication competence in both aspects.

Introduction

Disease treatments have been significantly influenced by the communications between patients, their families, and doctors,1,2 the lack of which may lead to malpractice allegations and complaints.3,4 In particular, inadequate communication may delay diagnosis and treatment. Therefore, for doctors, communication and interpersonal skills are as important as clinical skills and medical knowledge.

In the emergency cases, a qualified neurosurgeon should handle communication more competently than in other medical encounters. Ineffective communication and consultation before life-saving operations may postpone diagnosis and treatment, thus resulting in complaints and even litigation; in contrast, good communication may retrieve more detailed patient information and proper informed consent. Regardless, many neurosurgery residents remain unskilled in communication and interpersonal skills.

Doctor–patient communication competencies are being assessed by psychometrically sound communication assessment tools,5 mobile information and communication systems,6 database and multimedia authoring tools,7 and the Medical House Call tool.8 However, these techniques and tools can only be applied to patients capable of communicating. If the patients are not capable of communicating as they are admitted to the emergency room, coupled with the limited time, the strategies are prone to failure. Consequently, we created and trained a “standardized” family to simulate the communication counterparts and to evaluate, and rate the trainers. The standardized family consisted of actors without a medical background; they simulated patients’ families, or other representatives such as a friend, teacher, roommate, and/or colleague. When directly talking to the families of patients in a critical condition, above and beyond a kind attitude, well-developed and completely detailed medical practice and procedure information are important components of communication. In this study, we tried to develop a rating instrument that focused on the detailed content and knowledge used to inform patients’ families by referring to the Kalamazoo II Consensus Report.9

We intended to develop two detailed communication content checklists and a modified interpersonal skills inventory, aiming to evaluate the neurosurgeons integrity in the midst of communication skills assessments, to provide feedback for some participants, and to observe their communication competence in both aspects.

Methods

Participants

Four first-year and two second-year neurosurgery residents as well as six junior attending neurosurgeons from ten neurosurgery departments in east China, all men, aged between 25 years and 30 years, were voluntarily engaged in this study.

Assessment tools

Three assessment tools were applied in this study. One was an integrity checklist of communication content with 34 items and a 3-point rating scale for subarachnoid hemorrhage (SAH) scenarios; and the second checklist, with the same scale for head trauma (HT) scenarios (Tables 1 and 2). The third assessment tool was a 13-item modified version of the 5-point Likert Interpersonal Scale based on the US National Board of Medical Examiners prototype Clinical Skills Examination (Table 3).

Table 1

Integrity of the communication content with SAH patients’ families

Encounter	Undone(0–2 scores)	Insuff	Done(0–2 scores)
1. Introducing physician himself
2. Inform family members that the patient is in a critical situation and an emergency operation may be needed
Delivery of bad news and information of patient’s condition (3–8)
3. Possible diagnosis
4. Relationship between pathophysiologic procedures, symptoms, and outcomes
5. Briefly explain the Hunt-Hess scale and the relationship with outcomes
6. Prediction of mortality and morbidity rates according to the diagnosis and patient’s situation
7. Prediction of outcome and neurofunctions according to the diagnosis and patient’s situation
8. Possible improvement or deterioration at any time
Introducing risk factors associated with SAH (9–18)
9. Aneurysms rebleeding
10. Cerebral vasospasm
11. Hydrocephalus associated with SAH
12. Seizures associated with SAH
13. Hyponatremia or electrolyte imbalance
14. Aspiration pneumonia
15. Arrhythmia, myocardial infarction, and heart failure
16. Decubital ulcer and phlebothrombosis
17. Cushing’s ulcer
18. Other complications associated with SAH and unpredictable events
Informed consent and treatment options (19–28)
19. Necessity and purpose of emergency operation
20. Outcomes associated with operation options
21. Define modus operandi with neurosurgeon’s help
22. Introduce main operative procedures
23. Introduce the operator
24. Express neurosurgeon’s endeavor
25. Notify the rates and complications associated with anesthesia
26. Medicine, which is science other than witchcraft, cannot cure all diseases
27. Briefly introduce the postoperative treatment
28. Cost
Possible complications and rates associated with operation (29–34)
29. Complications associated with placement of femoral artery catheter
30. Complications associated with catheter cerebral angiography
31. Complications associated with aneurysm embolization
32. Complications associated with placement of stents
33. Other complications associated with interventional therapy and unpredictable events
34. Incompletely avoiding complications of operations

Abbreviations: SAH, subarachnoid hemorrhage; Insuff, insufficiency.

Table 2

Integrity of the communication content with severe HT patients’ families

Encounter	Undone(0–2 scores)	Insuff	Done(0–2 scores)
1. Introducing physician himself
2. Inform patients’ families of exigent communications and an upcoming emergency operation
Delivery of bad news and information of patient’s condition (3–8)
3. Diagnosis
4. Relationship between pathophysiologic procedures of trauma and symptoms and outcomes
5. Explain Glasgow coma scale and relationship with outcomes
6. Prediction of mortality and morbidity rates according to the diagnosis and patients’ situations
7. Prediction of outcome and neurofunctions according to the diagnosis and patients’ situations
8. Possible improvement or aggravation of pathogenetic condition at any time
Notify complications and risk factors (9–18)
9. Infections
10. Cerebrospinal fluid leakage
11. Cranial nerve injuries
12. Intelligence disadvantage
13. Hyponatremia or electrolyte imbalance
14. Pneumonia
15. Hydrocephalus
16. Decubital ulcer and phlebothrombosis
17. Post-traumatic epilepsy
18. Other complications associated with trauma and unpredictable events
Informed consent and treatment prescription (19–28)
19. Purpose of emergency operations
20. Outcomes associated with emergency operations
21. Define modus operandi with neurosurgeon’s help
22. Introduce main operative procedures
23. Introduce the operator
24. Express neurosurgeon’s endeavor
25. Notify the rates and complications associated with anesthesia
26. Medicine, which is science other than witchcraft, cannot cure all diseases
27. Postoperative treatment
28. Cost associated with operations
Rates and complications associated with emergency operations (29–34)
29. Intracranial hematoma
30. Neurofunctional impairment associated with operation
31. Infections associated with operation
32. Epilepsy associated with operation
33. Other complications associated with operation and unpredictable events
34. Incompletely avoiding complications of operations

Abbreviations: HT, head trauma; Insuff, insufficiency.

Table 3

Interpersonal skills inventory

	Disagree strongly	Disagree	Do not agree or disagree	Agree	Agree strongly
1. Physician introduced himself and let me know his role in therapeutic procedures.
2. The physician was warm and friendly throughout the procedure without being abrupt or impatient.
3. The physician listened carefully as I described my problem and did not interrupt me.
4. The physician encouraged me to ask questions.
5. The physician gave me adequate time to ask any questions and to express my concerns and opinions.
6. I could understand any technical or medical terms the physician explained.
7. The physician spoke clearly and precisely.
8. The physician did not lecture or talk down to me.
9. The physician showed interest, did not feel bored or ignore me when I spoke to him.
10. The physician appeared comfortable and at ease during the conversation.
11. The physician maintained appropriate eye contact with me throughout the conversation.
12. The physician ended the conversation appropriately and politely.
13. Overall, I felt comfortable with this physician.

Note: Disagree strongly to agree strongly: 1–5 scores.

Raters

Raters included one standardized family involved in communication; another observing communication; five professors (Jianmin Liu, Zhijian Yue, Xiaoping Zhou, Xiaowu Hu, and Yi Xu); and five associate professors (Yiqun Cao, Laixing Wang, Bo Hong, Qinghai Huang, and Wenyuan Zhao) from the Department of Neurosurgery, Changhai Hospital. They were all specially trained to identify and rate the participants’ performance by watching “full marks videos” executed by Dr Bo Hong.

Standardized family training

The standardized families, were trained to observe participants’ performance and score them accurately before the commencement of the research; they are briefly introduced in the Study design section.

Scenario 1: SAH

The virtual patient, Changxi Wang, described as a 44-year-old male smoker, chef, with hypertension and diabetes mellitus, was admitted to the emergency room because of the sudden onset of headache and coma. Emergency computerized tomography (CT) demonstrated diffused SAH, and CT angiography disclosed an aneurysm of the right posterior communicating artery; Glasgow coma scale 10 (E2V3M5),10 Hunt and Hess scale grade IV, and Fisher scale grade IV. The presumed primary therapeutic strategy was: (1) take medical measures to prevent rebleeding after SAH;11 (2) immediate interventional therapy after a cerebral artery digital subtraction angiography; and (3) possible lateral cerebral ventricle drainage depending on the CT post-aneurysm embolization.

The standardized family included a simulated patient’s wife: a 42-year-old tailor, Buddhist, introverted, middle school graduate, with a 16-year-old daughter. Standardized family behavior exhibited nervousness with great grief; the wife moved back and forth with her amulet while her legs jittered. She responded to participants politely and in a friendly manner.

Scenario 2: HT

The virtual patient, Xiaobao Ren, a 55-year-old woman, accountant, without chronic disease, was sent to the emergency room because of HT after a traffic accident. She had a Glasgow coma scale 7 (E1V1M5), right mydriasis (diameter 4.5 mm) without direct and consensual pupillary responses to light. Emergency CT showed a right frontotemporal contusion, laceration of brain, about 40 mL of subdural hematoma, and the middle line of the hemisphere shifted more than 1 cm leftward. The presumed primary therapeutic strategy was immediate evacuation of subdural hematomas.

The standardized family was a simulated patient’s son: a 28-year-old engineer, Catholic, extroverted and resolute, graduated from Fudan University (Shanghai, People’s Republic of China), and the only child. Standardized family behavior exhibited anxiety and impatience, clenching of fists, and asking for an emergency operation numerous times. He responded to the participants rapidly and concisely, but politely.

Study design

All participants were randomly paired into two groups, (Figure 1) with two first-year and one second-year resident and three junior attending neurosurgeons in each group. Two groups of participants encountered the SAH scenario as the first assessment. Then, one randomly chosen group received feedback immediately (the feedback group), whereas the other group was labeled as the no-feedback group. During the feedback, participants were allowed to read the items of the interpersonal skills checklist and the integrity checklists of communication content, and they acknowledged their scores. To exclude the impact of the first assessment, both groups were subjected to a second assessment with HT scenario 14 days later, during which no feedback was received from any groups. To evaluate any midterm impact of the feedback, both groups experienced a third assessment with SAH scenario 9 months later, during which the delivery of bad news, informed consent, and treatment prescriptions were required and included. Each group had to be finished in 20 minutes, otherwise the rating procedure was terminated. Assessment tools, raters, and standardized families in the third assessment were identical to those in the first SAH scenario.

Figure 1

Outline of study design.

Abbreviations: SAH, subarachnoid hemorrhage; HT, head trauma.

Data analysis

To determine the reliability level of the rating derived from the integrity checklists of communication and interpersonal skills checklists, inter-rater reliability and Cronbach’s α were calculated in the first SAH scenario. Communication scores of each scenario were processed by a two-way analysis of variance utilizing feedback (feedback, no feedback) and the education level of participants (eg, residents, junior attending doctors) as between-subjects variables (Table 4). We also launched a paired Student’s t-test on interpersonal skills and integrity of communication content in both groups between the first and second SAH scenarios (Table 5) to compare the midterm impact of feedback. SPSS 16.0 system (IBM Corporation, Armonk, NY, USA) was applied to perform statistical analyses, and P<0.05 was considered to be significant.

Table 4

Interpersonal skills and integrity of communication content comparisons

	df	MSE	F	P-value
IS aspect
SAH1
Group	1	0.028	0.008	0.929
Level	1	24.797	7.340	0.027
G × L	1	0.028	0.005	0.943
HT
Group	1	24.558	23.681	0.000
Level	1	5.558	5.359	0.049
G × L	1	0.926	0.893	0.372
SAH2
Group	1	10.704	7.593	0.025
Level	1	8.898	6.312	0.036
G × L	1	0.000	0.000	1.000
SAH2-SAH1
Group	1	9.630	7.204	0.028
Level	1	3.987	2.982	0.122
G × L	1	0.709	0.530	0.487
IC aspect
SAH1
Group	1	1.021	0.806	0.396
Level	1	311.780	246.120	0.000
G × L	1	0.083	0.066	0.804
HT
Group	1	7.787	2.202	0.176
Level	1	225.333	63.717	0.000
G × L	1	5.113	1.446	0.264
SAH2
Group	1	2.445	0.530	0.488
Level	1	178.225	38.604	0.000
G × L	1	5.672	1.228	0.300
SAH2-SAH1
Group	1	6.626	3.192	0.112
Level	1	18.542	8.933	0.017
G × L	1	4.380	2.110	0.184

Abbreviations: df, degrees of freedom; MSE, mean square error; IS, interpersonal skills; SAH1, first subarachnoid hemorrhage scenario; G × L, group times level; HT, head trauma; SAH2, second subarachnoid hemorrhage scenario; IC, integrity of communication content.

Table 5

Paired Student’s t-test between SAH1 and SAH2

	SAH1		SAH2		P-value
	Mean	Std	Mean	Std
IS
Feedback	57.10	2.72	60.00	1.73	0.031
No feedback	57.00	1.76	58.11	1.06	0.020
IC
Feedback	57.85	2.26	60.86	1.71	0.041
No feedback	58.43	2.37	59.96	2.11	0.005

Abbreviations: SAH1, first subarachnoid hemorrhage scenario; SAH2, second subarachnoid hemorrhage scenario; Std, standard; IS, interpersonal skills; IC, integrity of communication content.

Results

Reliability and validity of assessment tools

The reliability of the interpersonal skills assessment rating was acceptable (Cronbach’s α=0.752, intraclass correlation coefficient =0.987, 95% confidence interval [CI]), whereas the reliability of the integrity of communication content was good (Cronbach’s α=0.886, intraclass correlation coefficient =0.983, 95% CI). Both interpersonal skills and integrity of communication content assessment tools identified the differences between attending doctors and residents in all scenarios (Figures 2 and 3; Table 4), demonstrating the validity of those checklists by successfully distinguishing the two levels. In the first SAH scenario, the interpersonal skills checklists and the integrity checklists of communication content of the two groups did not differ significantly (P=0.929 and P=0.396, respectively), indicating a good homogeneity between groups.

Figure 2

IC scores in three scenarios.

Notes: Columns represent the mean IC scores of group consisting of residents’ and attending doctors’ scores. *Significant difference between levels in a scenario (P<0.05). No statistical difference was found between groups only levels in all the scenarios.

Abbreviations: IC, integrity of communication content; SAH1, First subarachnoid hemorrhages scenario; SAH2, Second subarachnoid hemorrhage scenario; HT, head trauma.

Figure 3

IS scores in three scenarios.

Notes: Columns represent the mean IS scores of group consisting of residents’ and attending doctors’ scores. *Significant difference between levels in a scenario (P<0.05). Significant statistical differences were found between groups in HT and second SAH scenarios. The performances of residents and attending doctors did not differ significantly in all scenarios.

Abbreviations: IS, interpersonal skills; HT, head trauma; SAH1, First subarachnoid hemorrhages scenario; SAH2, Second subarachnoid hemorrhage scenario.

Increase in communication competence by feedback

The feedback of the two groups differed significantly in the interpersonal skills in HT scenario (P<0.001), and remained thereafter until the second SAH assessment (P=0.025). In contrast, no significant differences of feedback existed in the two groups in the integrity of communication content in HT (P=0.176) and second SAH (P=0.488) scenarios.

Paired Student’s t-test revealed that both groups enjoyed boosted interpersonal skills and integrity of communication content after 9 months, which confirmed the sensitivity of both assessment tools. The increased integrity of communication content and interpersonal skills scores from the first SAH to the second SAH showed that the residents’ integrity of communication content scores rose faster (P=0.017) than those of the attending doctors, and the interpersonal skills of the feedback group was elevated more rapidly than that of the no-feedback group (P=0.028).

Discussion

Currently, doctors’ communication skills are mainly being evaluated by three methods:9 (1) checklists of observed behaviors during the interactions with real or simulated patients; (2) surveys of patients’ experience in clinical interactions; and (3) examinations using oral, essay, or multiple-choice response questions. This study selected the first protocol because: (1) it retains high fidelity to the evaluation on communication and interpersonal skills; (2) it minimizes the impact of raters;12 and (3) it allows standardized-patient exercises to be more reliable in evaluating history taking, physical examination, or communication skills than those in measuring problem-solving or clinical-reasoning skills.13,14

Although the Likert scale has been used in assessing communication skills for decades,15 it is concerned more with communication content, but it fails to cater to neurosurgeons in reality. The Likert scale was concerned more with doctors’ manners and attitudes, and patients’ sense, as well. A lot of medical disputes originate from the insufficiency of communication information as being against the rights and interests of patients and their families. Some doctors neglect to inform patients’ families, who may not be aware of a neurosurgical emergency scenario; this could be avoided by encouraging patients and their families to ask questions. Thus, patients’ families are accustomed to ascribing disease deterioration or unexpected events to a doctor’s incompetence,16 giving rise to conflicts confronting distrust and poor outcomes. To protect doctors from violence and litigation,17 protect patients’ rights and interests, and to minimize doctor–patient conflicts, we herein developed an integrity checklist of communication content as an independent assessment tool.

In the present study, these checklists were satisfactorily reliable according to their Cronbach’s α value and intraclass correlation coefficient. The validities of the two checklists could not be calculated in the absence of available gold standards for both communication attitude and contents. However, both assessment tools managed to identify the differences between attending doctors and residents (Table 3), concomitant with desirable feedback. Additionally, they met the demands of effective communication assessment, which includes seven elements, as follows: (1) building relationships, (2) starting discussion, (3) gathering information, (4) understanding the patient’s perspective, (5) sharing information, (6) reaching agreements on problems and plans, and (7) providing closure.18 Therefore, we believe the checklists are valuable, reliable, and sensitive.

The analysis of variance results of the scenarios (first SAH, HT, and second SAH scenarios) suggest that the feedback and no-feedback groups were almost the same at the threshold of the first SAH assessment, but they began to differ significantly regarding interpersonal skills in HT and the second SAH scenarios, respectively, after 14 days and 9 months. Nevertheless, the integrity of communication content did not differ significantly between the feedback and no-feedback groups in the three scenarios.

We ascribe the feedback effect on interpersonal skills to the capabilities of the participants per se in the first place. Their communication skills stemmed from previous experiences or observations of senior physicians even without specific training.19 Some of them may not have been aware of expressing their attitudes and behaviors appropriately at first, but were corrected by referring to the feedback. The feedback boosted interpersonal skills not only in the second SAH scenario (P=0.025), but also in the HT scenario (P<0.001), indicating that communication attitudes and manners can be applied to similar situations. Contrarily, the integrity checklist of communication content was designed to assess participants’ communication skills based on their knowledge and comprehension of a certain disease, every item of which consisted of considerable detail. Thus, the feedback may not suffice to exert discernible effect on the integrity of communication content.

We assumed that utilizing “full marks video” as feedback might help participants acknowledge their errors and correct their behaviors in the future. We also believe that the feedback did not function in the HT scenario owing to the communication differences between the SAH and HT scenarios. In other words, the integrity of communication content may be scenario dependent. After 9 months of clinical practice, both groups experienced significantly raised integrity of communication content scores (Table 5), verifying the enhancement of participants’ knowledge and comprehension of diseases.

Nevertheless, this study is still not impeccable. The bias of this study essentially originated from (1) the differences between rater sources, which had been subtly decreased by pre-assessment score training, however; (2) the slightly flawed integrity checklist of communication content, which may provide more levels to distinguish examinees’ performances in the case of a 5-point scale; and (3) the lack a sufficient list of disease and operation complications in the integrity checklist of communication content, which may have befuddled the raters.

We trained standardized families and enabled doctors to communicate with the patients’ families instead of the patients themselves, which maximally simulated the critical situation in emergency rooms and expanded the conception of doctor–patient communication. As to the integrity of communication, the rating protocol we developed herein emphasized the details of content and information of which we need to inform patients’ families, which successfully bridged the gaps induced by the sole attitude- and behavior-targeting assessments. Last, those experienced doctors enjoyed remarkably enhanced interpersonal skills within a short time, benefiting from the repeated and correct feedback.

Conclusion

Traditional doctor–patient communication does not give enough credit to communication content because it is more concerned about doctors’ manners and attitudes, as well as patients’ feelings. Neurosurgeons should communicate fully with patients and their families. The integrity of communication content is as equally important as doctor’s manners and attitudes, and can be evaluated simultaneously with interpersonal skills in communication competence assessment by these reliable, sensible, and feasible checklists. A primary feedback following evaluation may improve doctors’ interpersonal skills in different scenarios. In summary, both the integrity of communication content and interpersonal skill benefited from the method described herein in the midterm. Besides, the integrity of the communication content of residents ought to be further reinforced. Although we tried to avoid sensitivity to the Chinese culture in the design and application of the Standardized Family Model, more proof is still necessary.