Searching for a written patient feedback instrument for patient-medical student consultations.

OBJECTIVE: The Patient Teaching Associate (PTA) program at Eastern Health Clinical School uses volunteer patients with chronic illnesses in consultation-based medical student education. The PTA program aims to develop students' patient-centeredness and associated skills. Our study aims, 1) to identify key desirable characteristics of written patient feedback to doctors and/or students that focuses on patient-centeredness in consultations, and 2) to critically evaluate existing instruments to identify any suitable instrument for use for medical student teaching.
METHODS: We reviewed our experience with the PTA program and explored the literature on patient-centeredness and patient feedback to identify desirable characteristics of written feedback for our program. A systematic search was conducted to identify existing patient feedback instruments. These were then evaluated in light of criteria based on desirable characteristics.
RESULTS: Eight instruments met the inclusion criteria. While all were designed for patient use, none were ideal for the PTA program. The Doctors' Interpersonal Skills Questionnaire (DISQ), while not used with medical students, is the closest fit to criteria.
CONCLUSION: The lack of instruments specifically designed for written patient feedback to medical students highlights a gap in the current literature. PRACTICE IMPLICATIONS: The DISQ provides a good basis for developing a new feedback instrument focused on patient-centeredness in medical students.

Introduction

Having recognized the need for teaching patient-centeredness and active patient participation in medical education,1,2 Eastern Health Clinical School (EHCS) implemented a Patient Teaching Associate (PTA) program, modeled on the Patient Partner Program (P3) of the University of Tasmania.3 In the PTA program, patients who have chronic illnesses volunteer to engage in medical student–patient consultations and subsequently provide verbal and written feedback to the students. In contrast to role-plays and simulation, the encounter is with a real patient interested in the education of medical students – a PTA. The encounter is not scripted or rehearsed, and the patient has real signs, symptoms and lived experience. The PTA responds to the students, telling their own story and consenting to a physical examination. This places an emphasis on primary care in a rich social context and involves the PTA, a medically qualified tutor and a small group of student-peers.

In the PTA program, each student over the course of the year will have the opportunity to undertake each of three tasks (divided for convenience) of history taking, physical examination and management planning multiple times to develop their skills. Students receive verbal feedback from the tutor, patient and their peers in each consultation, with written feedback from the tutor and patient after the consultation.

The Medical Interview Satisfaction Scale (MISS-21) was used from the introduction of the PTA program as the written feedback instrument completed by patients. The MISS-21 has demonstrated reliability and validity in medical consultation settings.4 However, the instrument was not a perfect fit for the PTA program.

Feedback from our patient volunteers indicated that many of the items were not relevant. The MISS-21 was designed for general practitioner (GP) consultations. For example, the instrument was not applicable to the various roles students perform during the consultations, and the domains of stress relief and compliance intent were not pertinent. Additionally, as each patient completes multiple forms, with each taking at least 5 minutes, completing the MISS-21 takes too long in the context of the program.

In the PTA context, the design of a feedback instrument must cater for patients assessing medical students. In a typical PTA session, each patient has six forms to complete, three after each of two consecutive consultations. In our experience with patients using the MISS-21 form, patients regularly took >5 minutes per form, often requested help from the coordinator in interpreting the form and have not been able to comment on domains of distress relief or intention to comply with management as they are usually not relevant to student consultations.

The purpose of this study was to underpin research for the identification of an instrument allowing patients to provide efficient, meaningful and helpful feedback to the students in the context of developing patient-centered consultation skills. Our study aims, 1) to identify key desirable characteristics of written patient feedback to doctors and/or students that focuses on patient-centeredness in consultations, and 2) to critically evaluate existing instruments to identify any suitable instrument for use for medical student teaching. The ideal instrument would be reliable, valid and user-friendly in the PTA setting.

Methods

A two-stage literature search was undertaken. The initial search strategy focused on literature about patient-centered consultation skills and patients as educators to identify key desirable characteristics of written patient feedback to doctors and/or students. The subsequent systematic search focused on looking for existing patient feedback instruments that fulfill the purpose of providing feedback about patient-centeredness in consultation skills. Figure 1 presents a schematic description of our full methodology.

Figure 1

Schematic description of our methodology.

Abbreviation: PTA, Patient Teaching Associate.

The rationale for a two-stage literature search was so that the initial search would contribute to a refinement process to inform us about the relevant features we wished to capture in a feedback tool. This would allow us to determine the characteristics that would be vital for a feedback tool that both accurately assesses patient centeredness and meets the needs of the PTA program. In short, this process utilized the results of the initial search to define a set of literature-informed inclusion/exclusion criteria for use for the subsequent search.

Initial searches looking for useful characteristics of a patient feedback tool were performed using Medical Subject Headings (MeSH) and text words in the following databases: Ovid Medline, PubMed, the Cochrane Library and Google Scholar.

Table 1 lists the search terms used. The searches were independently conducted by HL and NH. Suitable articles were downloaded onto a separate database, and their relevance was assessed by studying the abstract. Disagreement was resolved by consensus between HL, NH and ES. Full-text articles of relevant papers were extracted, and their reference lists were also reviewed for additional articles that might be relevant. All searches were conducted in November and December 2014.

Table 1

MeSH and text words used for literature searches

Search	MeSH and text words
Stage 1 search to identify relevant characteristics of a patient feedback tool	“patient-centered care”“patient-centeredness”“patient-centered education”“patients as educators”“medical students”
Stage 2 search to identify feedback tools currently in use	“patient feedback”“medical students”

Abbreviation: MeSH, medical subject headings.

For the subsequent search for existing patient feedback instruments (Table 1 for search terms), we applied a two-stage inclusion/exclusion criteria method. This was done first to limit any existing patient feedback instrument to one that assesses patient-centered consultation skills as informed by the literature (and not just, e.g., clinical task competence) and second to determine the suitability of the existing patient feedback instrument for our PTA program. Table 2 lists the inclusion/exclusion criteria used for feedback instruments, and Table 3 lists the additional criteria to determine the suitability of the existing patient feedback instrument for our PTA program. All suitable instruments that met the criteria were reviewed by authors with extensive experience with the PTA program (CP and ES) with a view to implementation in the PTA program.

Table 2

Inclusion and exclusion criteria for feedback instruments

Inclusion criteria	Exclusion criteria
Must be completed by patients	Designed for completion by peers or observers
Must assess practicing doctors or medical students	Focused on organizational items (such as practice setting and administration)
Must provide individual performance feedback	Not used for individual feedback
Must assess key interpersonal skills	Did not assess interpersonal skills

Table 3

Additional criteria for applicability to the PTA program

Additional criteria for applicability to the PTA program
1. Easy/quick to complete (estimated at <5 minutes, without staff assistance)
2. Enables feedback on key features of patient-centeredness
3. Assesses integrated consultation fluency of medical student
4. Is externally validated

Abbreviation: PTA, Patient Teaching Associate.

Results

The initial search strategy focused on, 1) the nature of patient-centeredness, and 2) authentic patient feedback. Analysis of this literature resulted in the development of inclusion/exclusion criteria for the subsequent search (Table 2).

The subsequent search that was focused on looking for suitable patient feedback instruments returned a total of 1810 articles. This was narrowed down to 41 articles describing feedback instruments and their use. After applying our inclusion and exclusion criteria, eight patient feedback instruments remained, which met our criteria. These were further evaluated and described later.

Existing feedback instruments

The Chronically Ill Patients Evaluate General Practice (CEP)5 is a 51-item questionnaire for evaluating ten dimensions of care (Table 4). It was designed for chronically ill patients to evaluate general practice consultations and has a mix of organizational- and practitioner-level evaluation items. It is completed either immediately after the consultation or mailed to patients.

Table 4

Description of patient feedback instruments, surrounding data and relevance to the PTA program

Instrument	Scale and no. of items	Measurement	Data	Applicability to PTA*
Chronically Ill Patients Evaluate General Practice (CEP)5	6-point Likert scale, 51 items	Patient satisfaction with 10 dimensions of care: appointments/emergency availability, premises, continuity, cooperation, medical care, competence/accuracy, relation and communication, privacy, information and advice, support	No data on construct validity, factor analysis, reliable indices	1. No2. Yes3. Has not been tested4. No
Consultation Satisfaction Questionnaire (CSQ)6	5-point Likert scale, 18 items	Quantitative assessment of patient satisfaction. Four domains: general satisfaction, professional care, depth of relationship and perceived time	No correlation found between performance in video assessment of MRCGP exam	1. Yes2. Yes3. Has not been tested4. No
Doctors’ Interpersonal Skills Questionnaire (DISQ)7	5-point Likert scale, 12 items	Skills such as greeting, listening, explanations, abilities to elicit concerns or fears and respect shown to the patient	Reliable and valid instrument in assessing interpersonal skills. Positive feedback effect shown	1. Yes2. Yes3. Has not been tested4. Yes
General Practice Assessment Questionnaire (GPAQ)8	6-point Likert scale, 36 items	Five key areas:1) access, 2) technical competence, 3) effective GP communication, 4) GP interpersonal abilities, and 5) effective organization of care	Construct validity not specified directly	1. Yes2. Yes3. Has not been tested4. No
Medical Interview Satisfaction Scale (MISS-21)4	7-point Likert scale, 21 items	Looks at satisfaction with individual consultations including 4 subscales: communication and comfort, distress relief, compliance intent and rapport	Valid and reliable instrument for assessment of patient satisfaction	1. No2. Yes3. Has not been tested4. Yes
Medical Student Interviewing Performance Questionnaire (MSIPQ)9	5-point Likert scale, 14 items	Two subscales:• Rapport• Treatment feedbackIn addition, several items assessed overall perception of student’s interviewing performance	Data suggest that teaching in patient care and psychiatry is highly compatible	1. Yes2. Yes3. Yes4. Needs further testing
Patients Evaluate General/Family Practice (EUROPEP)10	5-point Likert scale, 23 items	Five dimensions of care – relation and communication, medical care, information and support, continuity and cooperation and facilities availability and accessibility	No data on construct validity or factor analysis	1. No2. Yes3. No4. No
Unnamed instrument described in “Patient feedback for medical students”11	6-point Likert scale, 11 items	Modeled off the core domains of:• The General Medical Council’s Good Medical Practice12• Medical students’ professional behavior and fitness to practice13• Tomorrow’s Doctors14	Large numbers of patients are needed to provide reliability	1. Yes2. Yes3. No4. No

Notes:

Key to applicability to PTA: 1, easy/quick to complete (estimated at <5 minutes, without staff assistance); 2, enables feedback on key features of patient-centeredness; 3, assesses integrated consultation fluency of medical student (see glossary); 4, is externally validated.

Abbreviations: PTA, Patient Teaching Associate; GP, general practitioner; MRCGP, Membership of the Royal College of General Practitioners.

The Consultation Satisfaction Questionnaire (CSQ) is an 18-item instrument developed in 1990.6 It quantitatively assesses patient satisfaction with visits to a doctor and is completed immediately after the consultation.

The Doctors’ Interpersonal Skills Questionnaire (DISQ)7 is a 12-item questionnaire that is designed to assess health professionals’ interpersonal skills in hospitals or general practice settings.

The General Practice Assessment Questionnaire (GPAQ) is a 36-item questionnaire for evaluating five areas of care.8 It is a shorter version of the original 53-item General Practice Assessment Survey (GPAS). The 36-item questionnaire comes in both a postal version, which is not visit-specific, and a specific post-consultation version.

The MISS-21 is a 21-item questionnaire.4 It assesses patient satisfaction with individual doctor–patient consultations. Patients completed the MISS-21 immediately after leaving the doctor’s consulting room.

The Medical Student Interviewing Performance Questionnaire (MSIPQ)9 is a 14-item questionnaire that assesses rapport and treatment feedback of medical students in a psychiatry setting. It is completed after the encounter with the medical student.

The Patients Evaluate General/Family Practice (EUROPEP) is a 23-item questionnaire for evaluating five areas of care.10 It contains organizational and practitioner elements that are assessed over the previous 12 months. It is completed either immediately after the consultation or mailed to patients.

The unnamed instrument described in “Patient feedback for medical students” by Lyons et al11 is an eleven-item questionnaire that assesses a student’s communication and professional behavior following an interaction with a patient. Patients completed the questionnaire immediately after the interaction with the medical student.

Evaluation of feedback instruments

After identifying the eight instruments related to patient-centeredness, we then sought to critically evaluate each instrument regarding suitability for the PTA program context. Additional factors used in selecting an instrument suitable for our PTA program relate to the particular objectives and context of the program. The last column of Table 4 outlines how these eight instruments measure up against these additional factors. Despite many of the above instruments having potential, the DISQ was considered the most suitable. While not tested with medical students, it is a reliable and valid instrument developed for assessing the interpersonal skills of GP trainees in the Australian context.

The CSQ and GPAQ were quick to complete and also assessed interpersonal skills. However, they did not assess patient perception of the clinical competence of a medical student and thus would not be relevant in our setting.

The CEP, EUROPEP and MISS-21 were excluded due to their length, which would not be realistic in the short time constraints of the PTA sessions.

The MSIPQ was not externally validated in this context, as it had only been used on psychiatric patients.

While the unnamed patient feedback instrument described in “Patient feedback for medical students” was specifically designed for medical students, it did not assess clinical competence, nor was it externally validated.

Discussion

Nature of patient-centeredness

Patient-centeredness is the philosophy that health care should revolve around the wants, needs and preferences of patients and is based on a deep respect for patients as whole human beings.15,16 Patient-centeredness is valued by both the medical profession and the patients due to the benefits in improving the quality of health care delivery.17 These benefits are well documented in the literature.1,18–20 Patient-centered care is guided by a respect for the patient’s preferences and values, with involvement of the patient and their family in shared decision-making. It is also important that there be clear communication and sharing of information among the health care team and between the health care team and the patient. Patient access to care and the coordination of care between health care providers are also valuable. These principles of the patient-centered approach lead to the benefits of improved patient satisfaction and patient empowerment, while improving efficiency of care and alleviating discomfort, anxiety and concern and reducing the perceived symptom burden.

Furthermore, as the population ages and chronic illnesses become more prevalent, caring for patients has become increasingly complex, requiring a multi-disciplinary team approach tied together by a patient-centered model.8 Many of these concepts are illustrated in the student behaviors that our patients target in their feedback to students and to staff. Patients are very aware of simple communication skills (e.g. clarity and speed of speech, appropriateness of terminology) and interpersonal skills (active listening, responding to cues, seeking elaboration). They often comment on whether a student “really understood me” or “cares about me”. Patients also regularly comment on students’ apparent clinical and social knowledge, their ability to integrate multiple points of information, their self-confidence and their ability to inspire confidence in the patient (which collectively we refer to as “integrated consultation fluency”).

One of the identified barriers to patient-centered care originates from a lack of emphasis on patient-centeredness during medical education.21 Moreover, research shows that empathy, which is essential to a patient-centered model, tends to decline throughout medical school.22 To address these concerns, it is crucial that patients be integrated into medical education as educators. Patients, especially those living with chronic conditions, are experts by experience and can provide unique learning opportunities and feedback to students. Several studies have implemented patient teaching models and found that students became more sensitive to the needs of patients, developed a stronger sense of respect for patients and became more confident in their own abilities.23–25 These studies also reported high student satisfaction with patient teaching.

Patient-centered care

Patient-centered care is a complex concept. Providing patient-centered care requires application of appropriate interpersonal and communication skills as well as the principles of patient centeredness previously identified.18 Patients are well situated to assess the interpersonal skills of doctors (or medical students).19 Interpersonal skills are an imperative part of a clinician’s armory and even impact on health outcomes, e.g. reducing blood pressure, controlling pain and reducing anxiety.26 Many patients rate these skills as being the key reason for choosing their physician.6 Studies of GP registrars show that these skills are best developed earlier in training,26 highlighting the need to include patient feedback in the education of medical students. During these teaching encounters, patients understood and felt satisfied with the contributions they were making to medical student education.20

Authentic patient feedback

Authenticity of the “patient voice” in medical education is an important underpinning tenet to developing patient-centered practitioners.23 Feedback is defined as being information about reactions to a person’s performance of a task, which is used as a basis for improvement.27 The aim of feedback (in the case of medical education) is to provide information to students with the intention of narrowing the gap between actual and desired performance.28 “For the junior doctor in medicine and surgery, it is a safe rule to have no teaching without a patient for a text, and the best teaching is that taught by the patient himself ” Sir William Osler, Address to the New York Academy of Medicine, 1903.29 Patient feedback has become an increasingly important component of multi-source feedback (MSF) assessment.30,31 While clinicians have the expertise to assess clinical performance, they are rarely present during other clinicians’ consultations;32 thus, the patient provides a unique insight into doctor–patient interactions. As MSF assessment demonstrates, it is advantageous to provide feedback from a variety of perspectives, which includes patients, along with the traditional reviews from peers and seniors.33 This can assist in harmonizing social and medical agendas.19 While there are many advantages to using patient feedback, limitations do exist. Patients are not medically trained and thus judge quality of medical care differently to medical professionals.19 For example, a medical educator might think a particular behavior is inappropriate, whereas the patient considers it acceptable.34 Additionally, patient feedback tends to measure expectations relating to the process of delivery of care, rather than achieving specific health gains, which could be expressed as consumer satisfaction.35 This also relates to patients focusing more on the time spent with them rather than the quality of medical consultations.36 Expectations are a driver of patient feedback and can be a reflection of a patient’s demographics.36 Patients were also more inclined to rate students favorably34 and give glowing reviews to doctors who were nicer to them.36 Furthermore, those who are chronically ill tend to give less favorable feedback than those with milder conditions.33

Design of appropriate feedback instruments

While the content of feedback is crucial, its effectiveness can be enhanced by its delivery. Feedback is best when given about specific behaviors and delivered in a timely manner.28 Also, alerting the learner to the criteria they are being assessed against makes the feedback more useful.28

As feedback is a vital element of medical education, feedback instruments have been created to improve the flow and ease of information transfer. Instruments can augment the feedback process by enabling a platform to collect and generate ideas.

Numerous instruments exist that aid in providing feedback. A recent review identified two instruments for use in family medicine practice that were dedicated to patient-centered care and a number of others that included some relevant measures.37 Some instruments were originally sourced from industrial settings and are now gaining acceptance in the health sector.32 As discussed, this review aimed to identify an instrument that was succinct, assessed interpersonal skills and the clinical abilities of a medical student and was reliable/valid.

Characteristics desired in feedback instruments

Feedback instruments should be easy to use and applicable to the intended nature of the feedback. It is important that individuals giving and receiving feedback understand which qualities are being assessed.32 Without this, feedback might not fulfill desired intentions. As discussed earlier, patient feedback is essential in creating a holistic review combined with reviews from peers and coworkers.36

Limitations of feedback instruments

In developing the ideal instrument, it is important to address and reduce limitations. While feedback is essential to medical education, many students and physicians can be resistant to receiving criticism.21 This can be overcome by utilizing it in a structured educational setting, such as a scheduled session/tutorial, where a more positive reception has been demonstrated compared to unstructured teaching in hospital wards.21 While grades may motivate students to work hard, they can also have an opposing effect, as students became anxious and stressed when they perceived this feedback was summative.38 When surveyed, students demonstrated a desire to receive feedback in a formative setting that stimulated them to reflect on what they were doing.13 Another drawback to ratings is the influence of familiarity between individuals giving and receiving feedback.32 This can be difficult to overcome as many doctor–patient relationships are built on increasing time spent together in consultations.

Design

Feedback can be delivered in many forms including verbal, written, person-referenced and task-referenced formats.39 It can also vary in the use of narrative or numerical scales. While numerical feedback has been the tradition, it may not adequately reflect the progress and competence developed by students in the complexities of medicine.39 Although most rating forms include sections for written (narrative) comments, they are often not fully utilized.39 Many broad statements such as “good job” or “average performance” are used in these sections, which do not identify areas for improvement.39 Many of our patients like to provide comments in addition to scoring items on the written instrument. The disadvantage of a free text design is the time-consuming nature of this process. Ideally, a feedback instrument achieves a balance between being efficient to complete and allowing for free comments.

A number of patient feedback instruments exist, although there is limited literature about the use of these instruments for patient feedback to medical students. The DISQ, a reliable instrument that assesses GP trainees’ interpersonal skills in medical consultations, was selected as meeting most of our requirements.

Strengths and limitations of the study

A strength of our study was that it involved extensive search of multiple literature databases with the search for suitable instruments being a systematic review. Furthermore, we used a “hybrid methodology” with the first stage of literature investigation informing inclusion and exclusion criteria for the instrument search.

A potential weakness is that the systematic review may have failed to identify all instruments with potential. The initial literature search was not systematic but, as part of the refinement process, we view it as apt for our purpose. The DISQ has not had its validity tested with medical students, but the authors considered that it had face value validity for our purpose warranting further development.

Further work

Some of the authors continue to work on the development of a variant of the DISQ for use with medical students in our program. This work, and subsequent pilot and validation studies, will be reported in due course.

Conclusion

This literature search summarizes information surrounding patient feedback in medical education and existing instruments that can be used to collect relevant and constructive feedback.

Patient-centeredness is a vital component of health care as it increases patient satisfaction and improves health outcomes. Thus, patient-centeredness must begin in medical school. In order to achieve this, patients should be utilized as educators. The feedback they can provide to students introduces a new perspective and allows for unique learning opportunities. While patients are not medically trained, they are consumers of medical services and provide a valid perspective on the doctor’s interpersonal skills and the fluency with which the consultation was carried out.

Many feedback instruments have been developed to measure the doctor–patient relationship; however, there are limited data surrounding their use with medical students. While many instruments were able to fulfill some of the criteria determined to be important for use in the PTA program, the DISQ instrument satisfied the most.

Practice implications

This literature search highlights that there are currently no patient feedback instruments that have been designed and validated specifically for medical students. While no such instrument exists, the DISQ provides a basis from which to develop a patient feedback instrument specific for medical students in the PTA program and other medical student consultation contexts.