Assessing patient-centered care: one approach to health disparities education.

BACKGROUND: Patient-centered care has been described as one approach to cultural competency education that could reduce racial and ethnic health disparities by preparing providers to deliver care that is respectful and responsive to the preferences of each patient. In order to evaluate the effectiveness of a curriculum in teaching patient-centered care (PCC) behaviors to medical students, we drew on the work of Kleinman, Eisenberg, and Good to develop a scale that could be embedded across cases in an objective structured clinical examination (OSCE).
OBJECTIVE: To compare the reliability, validity, and feasibility of an embedded patient-centered care scale with the use of a single culturally challenging case in measuring students' use of PCC behaviors as part of a comprehensive OSCE.
METHODS: A total of 322 students from two California medical schools participated in the OSCE as beginning seniors. Cronbach's alpha was used to assess the internal consistency of each approach. Construct validity was addressed by establishing convergent and divergent validity using the cultural challenge case total score and OSCE component scores. Feasibility assessment considered cost and training needs for the standardized patients (SPs).
RESULTS: Medical students demonstrated a moderate level of patient-centered skill (mean = 63%, SD = 11%). The PCC Scale demonstrated an acceptable level of internal consistency (alpha = 0.68) over the single case scale (alpha = 0.60). Both convergent and divergent validities were established through low to moderate correlation coefficients. DISCUSSION: The insertion of PCC items across multiple cases in a comprehensive OSCE can provide a reliable estimate of students' use of PCC behaviors without incurring extra costs associated with implementing a special cross-cultural OSCE. This approach is particularly feasible when an OSCE is already part of the standard assessment of clinical skills. Reliability may be increased with an additional investment in SP training.

BACKGROUND

Since Kleinman, Eisenberg, and Good's defining paper, Culture, Illness and Care (1978)1, patient-centered communication strategies have been taught as one approach to improving communication between physicians and patients2-5. Mead and Bower synthesized the empirical literature on the topic over several decades to describe a conceptual framework for patient-centeredness with five dimensions: “doing medicine” within a biopsychosocial perspective, eliciting details of the patient’s experience of illness, encouraging a sharing of decision-making power and responsibility, building a therapeutic alliance, and being sensitive to emotional cues6. In 2001, IOM's Crossing the Quality Chasm called for increased attention to patient-centered care (PCC) that was defined similarly as “providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions” (p. 6)7. Essential skills necessary for implementing a patient-centered clinical approach include the ability to elicit the patient’s personal story, to explore health beliefs and preferences, and to negotiate a management plan that is respectful of those preferences. In teaching these skills to medical students, Gomez and colleagues created a mnemonic, the 4 Cs, as a reminder of questions that could be used to help patients elucidate their preferences8:

Call: What do you call your problem?

Cause: What do you think has caused or contributed to your problem?

Concern: What concerns you most that we need to be sure to address?

Cope: What are you currently doing to cope with your problem?

Carrillo, Green, and Betancourt applied the patient-centered framework to the development of a cross-cultural curriculum, suggesting that providers could use the questions of the anthropologists and sociologists to better understand the needs of a diverse patient population4. The IOM report, Unequal Treatment, noted the value of cross-cultural education in understanding the role of social and cultural factors that may influence the delivery of health care and developing skills for improved communication, both potential factors in reducing the provider contribution to health care disparities7. The report recommended not only the implementation of cross-cultural educational programs focused on attitudes, knowledge, and skills for trainees and providers in medicine and nursing, but also the conduct of research to better understand the impact of such programs. A study by Beach and colleagues provides limited support to the idea that a patient-centered care approach can be effective in reducing racial and ethnic health disparities5. Medical students who reported positive attitudes towards PCC were scored significantly higher on history-taking, interpersonal communication, and counseling skills by African American standardized patients than were students without patient-centered attitudes. There were no differences in scores for the PCC and non-PCC attitude group when the standardized patients were white.

The purpose of the present study was to develop an evaluation tool with sufficient reliability, validity, and feasibility to identify the level of patient-centered behavior exhibited by trainees, here, senior medical students, during a standardized patient exam. Our goal was to go beyond the assessment of PCC attitudes to the measurement of actual PCC behaviors. A review of existing evaluation instruments by Mead and Bower in 2000 identified two major approaches to the assessment of PCC, self-report attitude surveys and coding systems for the analysis of verbal behavior6. Recent studies had reported the use of standardized patient examinations with cases specifically developed to represent a racial or cultural challenge to assess students′ cultural competency9–12. Drawing on the conceptual framework for PCC proposed by Kleinman and elaborated by Mead and Bower and recognizing the increasing use of standardized patients in assessing clinical skills13, we hypothesized that a checklist of PCC behaviors could be embedded in multiple cases across numerous medical problems and cultural contexts within an existing OSCE. This approach would provide an indication of the degree to which students demonstrated the behaviors, rather than self-reported attitudes, taught in the PCC cross-cultural curriculum, using existing OSCE cases. The questions we sought to answer were if this was a psychometrically sound approach and if it had any advantage in reliability, validity, or feasibility over the use of a single culturally challenging case in determining students′ use of PCC behaviors.

METHOD

This was a multi-institutional observational study of student performance on a measure of patient-centered care skills embedded in an existing comprehensive OSCE. IRB approval with an exempt status was obtained from both schools.

Participants

Subjects included all beginning senior medical students at two California medical schools located in the same culturally diverse community in the summer of 2006. School A had 160 students and school B had 162 students, resulting in a total of 322 participants.

Procedures and Instruments

OSCE examination

Both schools were members of the California Consortium for the Assessment of Clinical Competence and require their students to complete the same comprehensive OSCE as they transition from the third to the fourth year of medical school. In 2006, the OSCE included five non-emergency cases, which were used as the basis for this study. Each OSCE station consisted of a 15-min patient encounter in which the student independently interviewed a standardized patient (SP) in a simulated clinical setting. The SP used a checklist of essential items to score students′ performance of four skills: history taking, physical examination, counseling, and interpersonal communication. Cases and checklists were developed by Consortium members based on current evidence and consensus review. History-taking and counseling items were dichotomously scored as Done (credit) or Not Done (no credit). Communication items were rated on a 6-point scale from “unacceptable” to “outstanding.” Each case was played by two to three different SPs. Prior to the OSCE, the SPs underwent 16-20 h of extensive face-to-face training in standardizing the portrayal of the case and the calibration of the checklist items. Extensive SP guides were prepared to detail the patient’s story and provide specific guidance in how to determine whether or not a checklist item had been accomplished. The SPs were also expected to complete at least 10 h of self-study totaling almost 30 h of training before being certified to represent the case.

Embedded patient-centered care scale

The Embedded Patient-centered Care (PCC) Scale consisted of a total of 20 checklist items added to 4 non-emergency cases representing common primary care problems (back pain, cough, teenage counseling, DMII). The PCC checklist items were developed by two of the authors (DE, WM) based on Kleinman’s questions for exploring a patient’s experience of illness and developing a shared model to guide treatment planning1. The items were grouped into behavioral categories using the dimensions of the patient-centeredness conceptual model proposed by Mead and Bower 6 to help in examining curricular effects and providing feedback to students (Table 1).

Table 1

PCC Embedded Scale Individual Items Grouped Into Five Topic Areas

Five topic areas	SP rating of medical students on the 20-item PCC Scale itemsa	SP rating of medical students on the 21-item cultural challenge case itemsb
1. Meeting the patient’s needs (4 items total)	In four cases:	-The medical student worked toward a plan that addressed both the diagnosis and my concerns about my illness
	-The medical student “worked toward a plan that addressed both the diagnosis and my concerns about my illness”
2. Addressing feelings or concerns (4 items total)	In four cases:	-The medical student addressed my feelings: acknowledged and demonstrated interest in my expressed and/or unexpressed feelings and experience
	-The medical student addressed my feelings: acknowledged and demonstrated interest in my expressed and/or unexpressed feelings and experience”	-The medical student explored my fear of being experimented on
		-The medical student explored my concerns about having high blood pressure
3. Identifying health beliefs (4 items total)	In four cases:	-The medical student explored my belief that medicine need only be taken when symptoms are present
	-The medical student appropriately explored my perspective: encouraged me to identify everything that I needed to say”	-The medical student appropriately explored my perspective: encouraged me to identify everything that I needed to say
4. Exploring understanding (3 items total)	In two cases:	-The medical student explored my understanding of high blood pressure
	-The medical student asked what I thought was causing my problem(s)
	In one case:
	-The medical student found out my understanding of safe sex
5. Negotiating treatment plans (5 items total)	In four cases:	-The medical student negotiated with me a plan to help me comply with an area of difficulty
	-The medical student framed the action plan in such a way as to incorporate my beliefs and preferences
	In one case:
	-The medical student negotiated with me a plan to help me comply with an area of difficulty

aSP = Standardized patient

bThe 21-item cultural challenging case included 8 patient-centered care-oriented items as well as 13 content knowledge-specific items across history taking, counseling and communication components

Culturally challenging OSCE case

The cultural case was specifically developed to provide students with the challenge of blending a patient-centered with a disease-oriented approach in the context of a patient from an underrepresented minority group. Students were expected to recognize and then explore the health beliefs of an African American adult patient with poorly controlled high blood pressure. This culturally challenging case included 8 patient-centered care-oriented items as well as 13 additional content-specific disease-oriented history, counseling, and communication checklist items resulting in a total of 21 items.

Comparison of the embedded and single case assessment models

The 20-item Embedded PCC Scale reflected students’ use of patient-centered care skills in white patients, whereas the 21-item cultural challenging case reflected skills expected in managing a hypertensive patient from an underrepresented minority group.

Analysis

Descriptive statistics were calculated for the four clinical skill components, the culturally challenging case, the Embedded PCC Scale, and the PCC behavioral categories for each school individually. Independent sample t-tests were used to compare the scores between the two schools to determine whether the two samples could be combined for all subsequent analysis. Internal consistency of the PCC embedded items was calculated using Cronbach’s alpha. Initial results from a pilot test in 2003 were reviewed by the cross-cultural instructors in both medical schools to determine face validity14. Construct validity was addressed by establishing convergent and divergent validity using the total score for the culturally challenging case, counseling and communication (convergent), and physical examination skill score for the total OSCE (divergent).

RESULTS

Given the use of a similar approach to cross-cultural education at both medical schools, student performance on the OSCE as a whole and on the four skill scores at school A and B were almost identical. The differences that did exist were not statistically significant; thus, all further analysis was conducted on combined results.

Embedded PCC Scale

Medical students demonstrated a moderate level of patient-centered skill across the two measures: Embedded PCC Scale (mean = 63%, SD = 11%) and Single Culture Case (mean = 65%, SD = 12%). Embedded PCC performance varied across the five behavioral categories (Fig. 1). Students had the highest mean score for negotiating an action or treatment plan consistent with the patient’s preferences (mean = 66%; SD = 23%); however, the high standard deviation indicated a great deal of variability among students in the use of this aspect of patient-centered care skills. Students were least likely to explore the patient’s understanding or explanatory models (mean = 54%; SD = 23%). Items in this category included the exploration of the patient’s understanding of the specific problem–i.e., safe sex in the teenage counseling case. Once again, students varied widely in their use of this aspect of patient-centered care skills.

Figure 1

Descriptive statistics of the five topic areas, the Embedded 20-item PCC Scale, and the cultural challenge case scale. Note: The error bars represent 1 standard deviation above and below the mean.

Psychometric Performance of the Embedded PCC Scale

Psychometric properties of the Embedded PCC Scale can best be validated by comparing its results with that of the single cultural challenge case, a more common approach to the evaluation of cultural competency behaviors9–12.

Internal consistency

The reliability of the 20-item Embedded PCC Scale was 0.68 and was higher than that of the 21-item cultural challenge case (alpha = 0.60). As a general rule of thumb, the lowest acceptable level of alpha is 0.60 for exploratory purposes. Based on this assumption, both the Embedded PCC Scale and the single cultural challenge case demonstrated an acceptable level of internal consistency.

Validity

Without a gold standard for measuring patient-centered care, it is difficult to discuss validity. However, the items in the Embedded PCC Scale were determined to represent the dimensions underlying Kleinman’s questions for exploring the patient’s experience of illness after undergoing the scrutiny of several experts in the field of cross-cultural education following a pilot test of the Embedded Scale in 200311. Divergent validity was obtained by correlating PCC Scale performance to the full OSCE performance as well as two other general clinical skills components consisting of items that are disease-oriented (OSCE total, history taking, and physical exam). A low to moderate correlation would strengthen the argument that the Embedded PCC Scale is measuring something that is different from general disease-oriented clinical performance, which provides evidence for divergent validity. The Embedded PCC Scale did not correlate with the history-taking and physical examination component of clinical performance, which provided strong evidence for divergent validity (Table 2).

Table 2

Correlation Coefficients Between Embedded PCC Scales, Cultural Challenge Case, and OSCE Performance

	Embedded PCC Scale (20 items)	Cult. challenge case (21 items)
OSCE overall score	0.34a	0.17a
History-taking score	0.13b	ns
Physical exam score	ns	ns
Counseling score	0.41a	0.42a
Communication score	0.60a	0.32a
Cult. challenge case score	0.21a

aSignificant at 0.01 level

bSignificant at 0.05 level

ns: Not significant

The Embedded PCC Scale correlated moderately (r = 0.41 ∼ 0.60) and significantly with OSCE scores (p < 0.01) in the counseling and interpersonal communication components (Table 2). Although the correlation between the Embedded PCC Scale score and the culturally challenging case score (total of 21 items) was significant, the strength was weak (r = 0.21). These results in combination, the moderate correlations between the Embedded PCC Scale and the counseling and interpersonal communication components of the OSCE (which are largely linked to specific communication and counseling behaviors), along with a significant but weak relationship with the culturally challenging case, suggest that the PCC Scale is measuring similar but not identical constructs as those assessed in a comprehensive OSCE and the single case, which provided evidence for convergent validity.

Feasibility is concerned with the match between the intended purpose and the psychometric characteristics of the instrument. The Embedded PCC Scale demonstrated sufficient reliability for purposes of quality improvement and individual assessment in low stakes situations. The use of the PCC checklist that could be inserted into a regularly scheduled OSCE eliminated the need to develop a separate OSCE targeted only to the assessment of cultural competence. Finally, since the items were embedded in an existing comprehensive OSCE, the additional costs were minimal.

DISCUSSION

While it is not firmly established that a patient-centered care approach is associated with a reduction in health disparities, there is sufficient theoretical support for this effect, and training in providing patient-centered care is being recommended as one approach to improving communication and reducing health disparities. This study documents that one useful tool in measuring the effects of a patient-centered care curriculum is the insertion of checklist items representing the major constructs in patient-centered care into a comprehensive OSCE. A sufficient number of items can be achieved through the use of embedded items in multiple existing cases to produce an acceptable level of reliability without the cost associated with developing and implementing a separate cross-cultural OSCE.

Moderate correlations of OSCE scores in patient counseling and communication with PCC scores suggest that the Embedded PCC Scale measures a related but not identical variable as communication, providing evidence for convergent validity. The weak relationship between the Embedded PCC items and the cultural challenge case may imply that the two scales are measuring somewhat different things. One possible explanation could be that the analysis is based on all items within the cultural challenge case (including items that are not seemingly related to patient-centered care or cultural competency), whereas the Embedded PCC items are those specifically designed to assess patient-centered care skills.

The flexibility of the item structure in the Embedded PCC Scale makes it easy to apply across cases involving common problems in non-emergent situations. No additional stations need to be developed, and no additional cost is incurred in the development of the examination. The flexible nature of the PCC items, however, also carries with it ambiguity in its meaning. The PCC items presented a significant challenge in training the standardized patients to recognize the difference between a student who was telling him or her what to do in a nice way and actually negotiating for a treatment plan acceptable to both parties. Terms such as “addressing my feelings” or “appropriately explored my perspectives” could lend themselves to a variety of interpretation. The value of the checklist data is dependent on the accuracy of the SP’s performance. The case developers need to make sure that the wording of the checklist item as well as the intent behind that item is clear to the trainer and subsequently the SP. Only then will the SPs be accurate in performing and completing the checklist.

Limitations

Although the study was conducted at two institutions, the sample is still too small to be generalized to all medical student populations. The observation was done at a single point of contact, so predictive validity cannot be established. There was also a possible threat to internal validity where the use of more than a single standardized patient might have reduced the reliability index of the scale. However, this was not addressed in the current study.

Summary

A patient-centered approach has been suggested as one way to reduce disparities due to provider bias. The current study demonstrates that behaviors associated with patient-centered care can be reliably measured by adding a small number of relevant questions to the scoring checklist in each non-emergent case in a multiple-station OSCE. This approach is particularly feasible when an OSCE is already part of the standard assessment of clinical skills without having to develop new cases.