Assessment of Surgery Resident Competency Provided by Patients.

The objective of this study was to assess the competency of surgery residents from the patient perspective in the current healthcare environment in China. The authors performed an assessment of 508 surgery residents in Liaoning province. Seven patients were as a group to complete the self-administered questionnaires on the survey for each individual corresponding resident. A 5-point rating scale with an unable-to-evaluate category was used to assess surgery resident competency by patients. Reliability and validity were assessed by Cronbach alpha (α) and exploratory factor analysis, respectively. Statistical analysis was performed using SPSS 13.0. The surveys on 421 residents were valid, and the valid response rate was 82.8%. A total of 2947 questionnaires from patients were analyzed in this study. The Cronbach α coefficient was 0.92. The 4 factors emerging in the exploratory factor analysis reached a cumulative contribution rate of 66.98%. The items of "promotes health maintenance (talks about preventive care)" (206/7.0%), "tells me about any side effects of the medicine" (177/6.0%), "spends enough time with me" (189/6.4%), and "answers my questions thoroughly" (168/5.7%) were scored <4 by higher percentage of patients. The instrument provided an acceptable means for patients to evaluate the competency of Chinese surgery residents. Surgery residents should improve their competencies on preventive care, patient safety, and communication skills.

The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to train residents in 6 competencies, including patient care, system-based practice, interpersonal and communication skills, professionalism, medical knowledge, and practice-based learning, and to develop evaluation methods for assessing these competencies.[1] Previous studies have suggested that due to the lower levels of competency among clinicians, the quality of healthcare service in China is poor.[2,3] Low competency has also led to increased numbers of medical disputes and a crisis of confidence between doctors and patients in recent years.[4] To solve these healthcare problems in China, the National Health and Family Planning Commission of the People's Republic of China and the State Administration of Traditional Chinese Medicine of the People's Republic of China initiated a project in 2005 on “delivering patient-centered care to improve the quality of health service.” The assessment of ACGME core competencies provided by patients should be advocated to deliver patient-centered healthcare. A system-based practice may also provide optimal healthcare by developing a demonstrated awareness of and responsiveness to the larger context and system of healthcare.[5]

The relationship between the patient and professional has markedly deteriorated over the past decade in China[6-8] due to ineffective communication between the 2 parties.[9] Effective patient–professional communication promotes satisfaction,[10] and most importantly, facilitates patient adherence to treatment regimens.[11,12] However, interpersonal and communication skills are lacking in medical education in China.[13,14] Good interpersonal and communication skills enable a doctor to effectively demonstrate the acquisition and appropriate use of the other competencies, such as patient care, medical knowledge, and systems-based practice.[15] Based on the results of competency assessments, residents may increase their explanations to patients, change their communication strategies with nurses and peers, and improve their interpersonal and communication skills.[16-18]

Healthcare providers have high professional values that can enhance their job satisfaction and the quality of patient care.[19] Lombarts and his colleagues showed that physicians who had higher professionalism were also more likely to participate in professional quality improvement actions,[20] which in turn leads to better quality of care in the health services.[21,22] The adoption of professionalism is of prime importance because it shapes the identity of a profession and serves as a guide to action.[21,23] Significant attention has been paid to medical professionalism in residency training environments.[6,7] Although professionalism is receiving increased attention in Chinese medical education, the curricula designed to teach altruism, respect, compassion, honesty, integrity, and other humanistic behaviors are not being implemented quickly enough in China.[24,25] Compared with physicians, surgery residents are under greater pressure because surgery patients have more high-risk, acute, and severe illnesses; therefore, they are more prone to medical negligence and disputes.[26,27] Thus, competency must be assessed and developed more intensively for surgery residents.[28]

In this study, we assessed the competency of surgery residents from the patient perspective in the current healthcare environment in China. This analysis allowed us to better understand the current conditions, and in turn, provide several suggestions to health policy makers, healthcare administrators, and medical educators to improve resident competency.

METHODS

The survey used a random sample of 508 surgery residents from 7 (Dandong, Dalian, Yingkou, Chaoyang, Tieling, Fushun, and Shenyang) of the 14 cities that constitute the Liaoning province, covering the geographic diversity of the east, south, west, and north areas of Liaoning province. The cross-sectional study was performed in 15 hospitals from March to October 2014. Seven patients as an assessment group completed the survey for each individual corresponding resident using the self-administered questionnaires. The patients were surveyed in the process of completing the discharge formalities at the Admission Office of hospital. In our study, if 1 or more patients in the group completed below 80% of the questionnaire, the survey on corresponding resident would be considered as invalid; 2947 patient-assessment questionnaires on assessing 421 surgery resident competencies were analyzed, and were available for a mean of 7 per resident. The response rate was 82.8%.

The questionnaire included 2 parts, the demographic characteristics and the section assessing the resident competency. The sociodemographic characteristics of the surgery residents were provided by hospital administration department. The questionnaire from the Education Outcomes Service (EOS) Group of the Arizona Medical Education Consortium was developed for use by the patient.[29] This questionnaire consists of 23 items, with a 5-point rating scale (1 = never to 5 = always). A higher score indicates a higher level of competency.

All questions provided the respondents with the option of indicating that they were unable-to-evaluate the resident for that item. If the percentage of unable-to-evaluate items exceeded 10%, the item should be examined for revision or deletion. For each item on the instrument, the percentage of people who scored <4 over 5% was considered as poor competency. We used exploratory factor analysis to identify the factors for this instrument and to describe the relative variance accounted for by each factor. Reliability was assessed using Cronbach alpha (α) coefficient, which enables an assessment of the overall instrument stability. Statistical analysis used SPSS version 13.0 (SPSS Inc., Chicago, IL) for Windows.

The study received approval from the Bioethics Advisory Commission of China Medical University. The investigator informed all participants about the purpose of study and assured them of anonymity before the research was initiated. All participants gave written informed consent before completing the survey.

RESULTS

The questionnaire was effectively completed by 2947 patients, with a response rate of 82.8%. The sociodemographic characteristics of the surgery residents are shown in Table 1. Most of the residents were female (298, 70.8%). The mean age was 28.12 years (standard deviation: 2.56). For 60.1% (253), the highest level of education was equivalent to a Bachelor's degree, and the remaining (168) had a Master's degree or above.

TABLE 1

Sociodemographic Characteristics of the Surgery Residents (N = 421)

	n	%	Mean	SD
Age			28.12	2.56
Gender
Male	123	29.2
Female	298	70.8
Education level
Bachelor's degree	253	60.1
Master's degree and above	168	39.9
Work location
East Liaoning province	78	18.5
South Liaoning province	83	19.7
West Liaoning province	108	25.7
North Liaoning province	152	36.1
Work hours per day			8.51	1.32
Years of work experience
<1	241	57.2
1–2	102	24.2
>2	78	18.6
Monthly income, Yuan
<2000	226	53.7
2000–3000	138	32.8
>3000	57	13.5

SD, standard deviation.

The 23-item instrument showed a standardized Cronbach α of 0.92, indicating good internal consistency. The data met the Kaiser–Meyer–Olkin criteria for sample adequacy, with a result of 0.86, as well as Bartlett test of sphericity (χ2 = 572.16, df = 227, P < 0.0001). There were 4 factors on the instrument, including professionalism, patient care, system-based practice, and interpersonal and communication skills, which accounted for 66.98% of the total variance. The results of the factor analysis are shown in Table 2.

TABLE 2

Item Loadings for the 4 Factors of the Instrument From the Exploratory Factor Analysis (N = 2947)

Items	PR	PC	SBP	ICS
Promotes health maintenance (talks about preventive care, such as quitting smoking, weight control, alcohol, and exercise)		0.411
Asks regularly about prescription and nonprescription medicines I am taking		0.789
Clearly explains my medical problem(s)		0.804
Clearly explains my treatment choices		0.733
Tells me about any side effects of the medicine		0.798
Tells me when to return for follow-up care		0.847
Clearly explains how to avoid my problem(s) in the future		0.325
Demonstrates respect for my
Culture	0.848
Gender	0.753
Disability	0.827
Sexual orientation	0.732
Age	0.854
Religion	0.781
Is courteous to me	0.444
Listens to me				0.315
Spends enough time with me				0.129
Shows interest in my problems				0.570
Answers my questions thoroughly				0.597
Helps me with my fears and worries				0.492
Talks with me about treatment plans				0.718
Answers my messages in a reasonable amount of time				0.679
Refers to specialists when needed			0.498
Suggests community resources for additional information and support			0.396
% Variance (total = 66.98)	23.37	18.17	13.29	12.15

ICS, interpersonal and communication skill; PC, patient care; PR, professionalism; SBP, system-based practice.

The scores for majority of items were ≥4, as presented in Tables 3-6. However, in the aspect of “patient care” competency, the number of patients scored the items of “promotes health maintenance (talks about preventive care)” (206/7.0%), “tells me about any side effects of the medicine” (177/6.0%), and “clearly explains my treatment choices” (168/5.7%) <4 were relatively high. The items of “spends enough time with me” (189/6.4%) and “answers my questions thoroughly” (168/5.7%) were also scored <4 by higher percentage of patients. The respondents answered the majority of the items on the questionnaire. However, the “suggests community resources for additional information and support” (11.9%), “demonstrates respect for my disability” (11.6%), and “demonstrates respect for my sexual orientation” (14.3%) had unable-to-evaluate rates of >10%.

TABLE 3

Patient Descriptive Statistics and UE Rates of Patient Care (N = 2947)

Items	Score, Mean ± SD	Score < 4, n/%	UE, %
Promotes health maintenance (talks about preventive care, such as quitting smoking, weight control, alcohol, and exercise)*	4.59 ± 0.55	206/7.0	5.0
Asks regularly about prescription and nonprescription medicines I am taking	4.72 ± 0.52	88/3.0	3.6
Clearly explains my medical problem(s)	4.71 ± 0.56	88/3.0	5.0
Clearly explains my treatment choices*	4.69 ± 0.69	168/5.7	3.3
Tells me about any side effects of the medicine*	4.60 ± 0.63	177/6.0	3.6
Tells me when to return for follow-up care	4.72 ± 0.59	109/3.7	5.0
Clearly explains how to avoid my problem(s) in the future	4.75 ± 0.54	129/4.4	3.8

SD, standard deviation; UE, unable-to-evaluate.

*High percentage of patients who scored the item <4.

TABLE 6

Patient Descriptive Statistics and UE Rates of Professionalism (N = 2947)

Items	Score, Mean ± SD	Score < 4, n/%	UE, %
Demonstrates respect for my
Culture	4.89 ± 0.32	50/1.7	5.2
Gender	4.83 ± 0.40	50/1.7	7.4
Disability*	4.89 ± 0.40	68/2.3	11.6
Sexual orientation*	4.85 ± 0.38	130/4.4	14.3
Age	4.84 ± 0.40	68/2.3	6.2
Religion	4.86 ± 0.37	145/4.9	7.6
Is courteous to me	4.71 ± 0.53	50/1.7	3.6

SD, standard deviation; UE, unable-to-evaluate.

*UE rate is >10%.

DISCUSSION

In this study, we evaluated surgery residents for competency from the point of view of the patient. The exploratory factor analysis results showed that 4 factors emerged, including professionalism, patient care, system-based practice, and interpersonal and communication skills, accounting for a total of 66.98% of the variance. The internal consistency reliability analysis (Cronbach α) suggested that the instrument of evaluation was internally consistent. Thus, our study confirmed that the instrument of evaluation was a valid and reliable tool to measure competency among Chinese surgery residents from the patient perspective.

With regard to competency on patient care, the number of patients scored the competency for preventive care (206/7.0%) was higher. Preventive healthcare reduces the disease incidence,[30-32] which is considered as more cost-effective than a curative strategy.[32,33] However, in China, the utilization of preventive healthcare is challenging for many people, particularly those living in rural areas and those without medical insurance, because preventive healthcare activities are associated with socioeconomic conditions.[32] Additionally, 1 policy, the New Cooperative Medical Scheme initiated in 2003, funds only a few preventive services, which may lead to a lower focus on preventive care.[30] Regarding another factor of patient care, many patients (177/6.0%) regarded resident competency in relation to drug side effects as poor. A large number of people are injured or die in hospitals due to adverse drug events each year,[34] and the delivery of unsafe medical care has resulted in a trend toward increasing conflict between physicians and patients in China.[35] Thus, patient safety should have particular importance,[36,37] and clinical pharmacy services in particular can contribute to improving patient safety and reducing hospital costs associated with medication errors.[38] These findings suggest a need for surgery residents to focus on preventive care and raise concerns regarding patient safety.

As an important component of system-based practice, our study also found that the unable-to-evaluate percentages of competency related to suggesting community health resources were over 10%. Although community health service (CHS) organizations have become essential parts of the primary healthcare institutions, CHS is relatively underdeveloped in urban areas.[39,40] There are existing bottlenecks in the sustainable development of CHS in China, such as the inequitable distribution of healthcare workers.[39] So people's awareness of CHS is not strong. With the current inadequacy of health resources in China, it is necessary to further develop CHS to provide additional information and support to patients under government guidance.

Our results indicate that surgery residents should pay more attention to spending enough time with patients and answering questions thoroughly. Based on the findings of Salib et al, the majority of participating residents regarded various communication skills, including the duration of communication with patients, to be highly important in patient care.[41] Thus, teaching communication skills at all levels of medical education should be promoted, and additional training in communication for healthcare workers is necessary.

Although all patients could answer the majority of items on the questionnaire, there were specific types of items that showed higher unable-to-evaluate percentages. These items tended to focus on aspects of professionalism, such as respecting the patient's sexual orientation. Besides the heterosexuality, discrimination against other sexual orientations exists in China.[42] Due to social stigma, respondents may have declined answering the related question about sexual orientation to protect their privacy. In China, along with advances in health sciences, increasingly complicated working conditions have led to additional ethical dilemmas.[22] Particularly in recent years, workplace violence against medical workers has become more common and serious. These changes make the evolution of professional values more important and have drawn greater attention to these issues.[43] Effective strategies should be proposed to facilitate Chinese physician professionalism,[8] and professional development programs and medical ethics education should not only emphasize the general principles involved but also formulate guidelines on how these principles can be achieved in practice.[44]

Several limitations of the study should be noted. First, our study used a cross-sectional design; therefore, no causal conclusions can be drawn. In addition, the subjects were recruited from several cities in Liaoning province, and they may not be representative of all surgery residents in the entire country. Future studies should examine surgery residents in a larger sample size to enrich the data. The “top box” existed in our study. One possible cause may be that the standardized training and evaluating system for Chinese residents have been increasingly improved to enhance the quality of health service in recent years, so the scores on some items were high.

CONCLUSIONS

The instrument of evaluation was acceptable for patients to evaluate the competency of Chinese surgery residents. To improve surgery resident competency, it is necessary to place greater emphasis on enhancing competency on preventive care and raising concerns regarding patient safety. Moreover, training in communication for healthcare professionals is necessary.

TABLE 4

Patient Descriptive Statistics and UE Rates of System-Based Practice (N = 2947)

Items	Score, Mean ± SD	Score < 4, n/%	UE, %
Refers to specialists when needed	4.60 ± 0.72	118/4.0	6.6
Suggests community resources for additional information and support*	4.12 ± 0.33	395/13.4	11.9

SD, standard deviation; UE, unable-to-evaluate.

*UE rate is >10%.

TABLE 5

Patient Descriptive Statistics and UE Rates of Interpersonal and Communication Skills (N = 2947)

Items	Score, Mean ± SD	Score < 4, n/%	UE, %
Listens to me	4.64 ± 0.63	50/1.7	5.0
Spends enough time with me*	4.52 ± 0.62	189/6.4	3.6
Shows interest in my problems	4.70 ± 0.48	109/3.7	5.0
Answers my questions thoroughly*	4.68 ± 0.52	168/5.7	3.8
Helps me with my fears and worries	4.73 ± 0.55	68/2.3	5.0
Talks with me about treatment plans	4.71 ± 0.50	68/2.3	5.2
Answers my messages in a reasonable amount of time	4.70 ± 0.52	109/3.7	5.2

SD, standard deviation; UE, unable-to-evaluate.

*High percentage of patients who scored the item <4.