Critical patient insights from the same-day feedback programme at Stanford Health Care.

PATIENT-CENTERED ORGANISATIONS: Healthcare organisations now integrate patient feedback into value-based compensation formulas. This research considered Stanford Healthcare's same-day feedback, a programme designed to evaluate the patient experience. Specifically, how did patients with cancer interviewed in the programme assess their physicians? Furthermore, how did assessments differ across emotional, physical, practical and informational needs when interviewed by volunteer patient and family partners (PAFPs) versus hospital staff? PATIENT-PHYSICIAN COMMUNICATION BARRIERS: Integral to this research was Communication Accommodation Theory (CAT), which suggests individuals adjust interactions based on conversational roles, needs and understanding. Previous influential research was conducted by Frosch et al (2012) and Di Bartolo et al (2017), who revealed barriers to patient-physician communication, and Baker et al (2011) who associated CAT with these interactions. However, we still did not know if patients alter physician assessments between interviewers. VOLUNTEERS COLLECT PATIENT NEEDS: This mixed methods study worked with 190 oncology unit patient interviews from 2009 to 2017. Open-ended interview responses underwent thematic analysis. When compared with hospital staff, PAFPs collected more practical and informational needs from patients. PAFPs also collected more verbose responses that resembled detailed narratives of the patients' hospital experiences. This study contributed insightful patient perspectives of physician care in a novel hospital programme. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Background

Stanford Health Care (SHC) created the same-day feedback (SDF) programme in June 2009 to evaluate patient experiences. From June 2009 to May 2013, SDF members were SHC hospital staff; from June 2013 to June 2017, SDF members were volunteer patient and family partners (PAFPs). PAFPs were former patients or relatives of former patients.1 2 Both hospital staff and PAFPs received 1 day in-person training on semistructured bedside interviews to assess patients’ hospital experiences; training also included facilitated presentations on administrative systems access, interview best practices, common patient questions and helpful hospital resources.1 2 It was hoped that the programme’s transition to PAFPs could improve patient feedback. The effects of this transition were central to this research. To date, SDF and its outcomes have not undergone mixed-methods analysis to explore how patients interacted with the different interviewer groups, and if this led to significant variations in collection of patients’ needs and experiences.

A review of the literature revealed that the needs of patients with cancer fell into four categories: emotional, physical, practical and informational.3 It is known that volunteers can help meet these needs,3 but how well they identify them in comparison with hospital staff is what this research aimed to find out. SDF’s transition from hospital staff to PAFPs was suitably structured to provide insight into how patients might have adjusted messages based on the role of the interviewer. It has been hypothesised that healthcare participants adjust their dialogue for communication with different groups based on Communication Accommodation Theory (CAT).

At the junction between linguistics, communication and social psychology, CAT is a ‘framework for understanding the interpersonal and intergroup dynamics of speakers adjusting their language and non-verbal patterns to communicate’.4 The theory focuses on when, how and why speakers adjust their messages4; adjustments might then be made based on trust, respect and acknowledgement of social differences. Factors in conversational outcomes that relate to the healthcare system include the participant’s status, role relations and emotional and relational needs.4–9

CAT has been documented in the healthcare setting. One study found that patients disclosed more information to female physicians,10 while another study found that Dutch doctors had longer and more involved conversations with Dutch patients compared with immigrant patients.11 These findings revealed that patients and clinicians may alter communication styles based on audience.

Physicians are directly involved in coordinating patient care throughout the healthcare team, and where research has shown lapses in physician communication, it was hypothesised that patient assessments of this population might provide rich sources of information.10 12 13 The SDF programme allowed real-time insight collection from patients regarding their physician assessments. This differed from after-visit satisfaction surveys such as those from Press Ganey as the programme provided the opportunity to respond to and resolve concerns while the patient was still within the hospital’s care.1

Measurable interview differences in emotional, physical, practical and informational needs from physician assessments between hospital staff and PAFP interviewers are potentially significant to healthcare economists, policymakers and medical administrators who may decide to incorporate similar programmes. Thus, this research aimed to determine if CAT is applicable to needs collection by examining if patients alter feedback when interviewed by hospital staff versus PAFPs in a single hospital oncology ward study.1 2 Feedback may identify areas of improvement that could later be addressed by the healthcare team.

Design

Part 1: population sample and selection

Interview responses were provided by the SHC Service Excellence Department. Our sample includes 134 hospital staff interviews (June 2009–May 2013) and 56 PAFP interviews (June 2013–June 2017) from the primary oncology unit; one interview from staff and four interviews from PAFPs were incomplete and therefore could not be included in this study (see table 1). Patients were admitted 24 hours in advance of being interviewed. We focused on the open-ended questions about physicians as this material could be analysed for the patients’ own words about their direct experiences regarding emotional, physical, practical and informational needs (see online supplementary appendix A).3 The oncology unit was chosen because it has participated in SDF since its 2009 launch. Additionally, patients in this unit have larger care teams which are necessary for more complex care. Stanford University’s institutional review board determined that the present study was exempt from review and waived the need for participant informed consent (see online supplementary appendix B).

Table 1

SDF programme: patient demographics

Patientinterviewer	Samplesize	Hospitalunit	Serviceline
Hospitalstaff	n=134	CancerCentre	(49)(36)(9)(6)	OncologyHaematologyInternal Med.Other
PAFPs	n=56	CancerCentre	(17)(28)(4)(7)	OncologyHaematologyInternal Med.Other

PAFPs, patient and family partners; SDF, same-day feedback.

Part 2: interview structure

From June 2009 to May 2013, staff conducted an average of 2.85 interviews per month; from June 2013 to June 2017, PAFPs averaged 1.16 interviews per month. Interviewers spoke English. Both hospital staff and PAFPs received interviewer, administration and ethics training.1 2 SDF members visited in-patients on weekdays, independent of the healthcare team, and used a semistructured bedside interview to assess patient hospital experience. Interviews in both groups lasted 15 to 60 min, with an average of 25 min. Patient interviewees were children and adults.

Part 3: data organisation and analysis

Qualitative

Internal and external code generation followed Directed Content Analysis, whereby literature review of CAT and related healthcare communication research helped create 34 codes defined before and during analysis (See online supplementary appendix C).4 5 11 14 These codes were then sorted into the four main needs as described below (See figure 1). Coding was completed independently by two researchers (AL, US). A third researcher (AP) was asked to analyse data where coding differences could not be reconciled, however the initial coders reached full agreement. To ensure consistency between coders, Cohen’s Kappa coefficient was calculated for these four main themes. Kappa coefficient is a measure of agreement that takes random variation into account.15

Figure 1

Flow chart for the research steps of implementation. AIC, Akaike’s Information Criterion; PAFPs, patient and family partners; SDF, same-day feedback.

Written analytic memos were completed after each coding round to interpret themes developing in the data. Noting the patterns and resultant themes was a method of extracting data; exploring contrasts and comparisons tested the conclusions and practical significance across both groups.16 The 34 codes were then matched based on thematic similarity to emotional, physical, practical and informational needs.17 To verify conclusions drawn from matrix building and thematic analysis, two tactics were used: looking for negative evidence and making if–then tests. First, looking for negative evidence was a natural complement to the previous method of drawing conclusions through patterns. Outliers and rival explanations were actively sought in the interview responses to disconfirm findings.16 Second, making if–then tests formalised propositions for testing in the responses.16 For example, ‘If a patient response is marked with the in vivo code ‘Physicians are Excellent,’ then they are more likely to have been interviewed by hospital staff’.

Quantitative

We first tested statistical significance of emotional, physical, practical and informational needs between groups. SAS V.9.4 QQ plots revealed that categorisation of needs was normally distributed and called for paired t-tests. We then tested statistical significance of response word counts and qualitative codes assigned to each group. SAS V.9.4 QQ plots revealed that descriptive and interpretive coding data were not normally distributed.18 Non-parametric Wilcoxon two-sample tests were then used via ‘PROC NPAR1WAY’ to determine if the differences in medians and interquartile ranges between groups were statistically significant.19 The Benjamini-Hochberg procedure corrected for the above multiple correlated tests.20 21 Based on previous healthcare literature, the false discovery rate was set to 10% for this study.22 23

Lastly, we tested multiple linear regression models of qualitative codes as parameters attributing to patient response word counts using Akaike’s Information Criterion (AIC) via ‘PROC PHREG’. AIC has been shown as a practical method to examine collected data in clinical settings.24 A significance level of 0.25 was used to specify entry of a qualitative code into the model and 0.05 specified the significance level required to remain in the model. This final quantitative step connected the qualitative codes with patient response word counts and explored the possibility of interviewer groups having a positive or negative effect on word count.

Part 4: public and patient involvement

Public involvement was crucial to the success of this study. Former patients and their family members who volunteered to be a part of the PAFP programme interviewed hospital in-patients to gain insights into the healthcare experience. The interview methodology was developed by advisory councils with patients, families and interviewers to cover as many aspects of the hospital experience as possible. In addition, independent conversations with four experienced PAFPs about their programme involvement aided in development of research objectives and codes.

Results

Qualitative

Coding themes and subthemes are shown in diagrammatic form (see online supplementary appendix D). Examples of each code are also shown (see online supplementary appendix C). Average Kappa coefficient calculated for all codes was 0.66. While interpretation varies based on the context of assessment, a value of 0.66 generally indicates substantial agreement.15 Since the patient response data from interviewer groups were sorted into emotional, physical, practical and informational needs (see figure 2), the subsequent findings have been organised first by these four needs and then by interviewer group in accordance with the cross-case analysis inherent in the study design (see figure 3).

Figure 2

Sorting of codes into the four needs.

Figure 3

Appearance of qualitative codes as a percentage of total assigned codes in each group. PAFPs, patient and family partners.

Emotional needs

A large component of emotional needs was the descriptive code Compliment, as 133 out of 190 patients responded with a compliment. A noticeable trend emerged as the two-coder groups categorised responses as Compliment.

Staff

When interviewed by staff members, patient responses frequently included a brief phrase of approval that included one of four words: good, great, excellent or wonderful. These phrases accompanied complimentary brevity in patient responses. For example, when asked to assess their physicians, patients interviewed by staff members responded, ‘My doctors are great’, ‘They are good’ or ‘The physicians are excellent’.

Patient and family partners

Patients interviewed by PAFPs connected their compliments to their satisfaction with physicians and had a focus on relationships. In contrast to patients interviewed by staff, those interviewed by PAFPs often mentioned the social nature of their physicians. For example, ‘The best thing about them is that they fulfil the medicine part of their job as well as the human aspect of it’.

Physical needs

The most distinguishable coding trend in physical needs was NarrativeOfCare: instances where the patient provided a detailed account of his or her hospital experience in relation to physicians, thereby integrating characters, actions and outcomes to form a story. For example:

I trust my doctors. They are knowledgeable. Last year I had been going to my primary care physician and complaining about fatigue and feeling generally sick. My PCP didn't do much. When I finally came to [Location] they immediately found that I had cancer. I was in the ER and after one blood test they knew what was going on. At that time, I stayed here for 3 months. I was out of the hospital for almost a year before I had to come back. I'm here for a week of chemo this time. A few weeks after that I'm scheduled for a bone marrow transplant. I'm anxious about it, but I really trust my doctors with what they are doing and what they say.

The patient narratives were primarily positive, explaining how physicians exceeded expectations or providing specific examples of care. Patients tended to include details of diagnoses, diseases and medical tests associated with their hospital experience.

Responses in this group included complimentary elements such as a focus on relationships or trust, and ‘being in good hands’. These specific types of responses were absent from NarrativeOfCare collected in the staff group.

Practical needs

The code with the most obvious difference between interviewer groups was Treatments.

The responses associated with Treatments occurred at a significantly lower frequency and were extremely brief. Other codes were similar when compared with PAFPs. For example:

The doctors are great! They have been explaining everything to me. They keep me updated on the progress of my treatment and they provide explanation why progress is delayed at times.

The PAFPs collected responses categorised as Treatments were twice as prevalent (see online supplementary appendix E). These comments also referenced the relationship aspect of treatment in addition to the technical aspects such as medication, a trend that was missing in staff interviews. For example:

I know my doctors very well as they have been the same team of doctors caring for me since the beginning of my treatment. I have nothing but high regards for all of them. They are very informative and treat me like a human being. I didn't have any clear set of expectations for my care providers since I didn't have a whole lot of experience in a hospital prior to getting diagnosed with cancer. However, the doctors have definitely exceeded any expectations I did have.

Informational needs

Findings for Informational Needs were similar to those of Practical Needs. While many of the codes were alike, TeamCards and Talking were more frequent and detailed for PAFP (see online supplementary appendix E). The former code refers to a small paper sheet given to patients that contained physician information. For example, one patient interviewed by PAFP said:

The doctors have been good but they do always seem like they are in a hurry. They come in and see me and then they have to leave to see other patients. I know that they have to do that, but it always seems pretty rushed. I believe they gave their team cards to my wife, but I'm not sure. It's always hard in a teaching hospital because so many people come in with the doctors, but the head doctor seems to have that well under control. One thing I would like to know is when they are going to come around, especially the residents. It would help to have a heads up.

One finding in the PAFP group that did not occur in the staff group included disappointment with TeamCards. TeamCards became an in vivo code after 13 references appeared in each interviewer group. Two patients interviewed by PAFPs suggested the cards were unhelpful. These patients said, ‘I’ve been getting the team cards but they have no contact information on them, so I don't know if they're that useful’, and ‘I have their team cards, but they seem kind of useless because they don't have any contact information on them’. Barring this finding, the PAFP group followed results established in the staff group.

Quantitative

T-test revealed two statistically significant differences between PAFP and staff groups in needs collection (see table 2). The PAFP group identified practical and informational needs more often than their staff counterparts and in other areas the results were not statistically significant.

Table 2

T-test of the average mentions of the four kinds of needs per one patient comment

	PAFP	Staff	∆	P value
Emotional	2.25±1.21	2.14±1.09	0.11±1.13	0.55
Physical	2.00±1.29	1.70±1.28	0.30±1.28	0.15
Practical	0.55±0.66	0.23±0.50	0.32±0.55	0.0015*
Informational	1.79±1.37	1.37±1.05	0.42±1.15	0.0431*

*Statistically significant.

Wilcoxon two-sample tests and multiple testing correction by the Benjamini-Hochberg procedure revealed 11 statistically significant coding differences between PAFP and staff groups. The PAFP group had more prevalent use of codes Courteous, NarrativeOfCare, Optimistic, Satisfied, System, Talking, TeamCards, Travel, Treatments, Trust and higher response_WordCount (see online supplementary appendix E).

Akaike’s Information Criterion (AIC) generated a model of six independent variables for response_WordCount scalar dependent variable with an R2 value of 0.669 (see table 3).

Table 3

AIC model for predictors of patient response word counts

Parameter	Estimate	SE	T value	Pr>|T|
Intercept	29.49	4.03	7.31	<0.0001
Disease	48.10	8.44	5.70	<0.0001
NarrativeOfCare	46.79	4.60	10.18	<0.0001
TeamCards	28.37	4.98	5.70	<0.0001
System	27.80	5.01	5.55	<0.0001
Treatments	14.82	3.83	3.87	0.0002
Courteous	11.20	4.48	2.50	0.0132
R2	0.669257

AIC, Akaike’s Information Criterion.

Discussion

Qualitative and quantitative analyses were combined to determine how patients in different interviewer groups assessed their physicians. Our main finding, supported by both qualitative and quantitative analyses, is that interviews conducted by PAFPs are better at the identification of practical and informational needs. The ability to recognise a lack of information is important because it increases patient satisfaction. As expressed in a patient response from this study, ‘informed patients are happy patients’. When communication was clear and reliable, patients were satisfied; when patients were not informed, problems arose. This study’s findings are also consistent with a previous conclusion by Kaplan et al that care teams can ‘add value by talking more’, referring to open lines of communication.25 This same need for clear reliable communication was echoed across social media regarding clinical trials and in multiple systematic reviews in social care and research design.26

Word count was an important secondary outcome measure as a higher word count might suggest an in-depth interview with potential for more meaningful feedback.27 A closer look at the data shows that patients used more words when sharing stories of their hospital stay (NarrativeofCare), thus providing a deeper insight into the patient experience. The difference in the presence of the code NarrativeofCare between PAFPs and hospital staff was statistically significant both quantitatively and qualitatively, with PAFPs having a greater occurrence of NarrativeofCare. These patient narratives provided insight into their lived hospital experiences, critical for inhouse needs finding as well as understanding how they interacted with and assessed their physicians.

NarrativeOfcare and associated response word counts could be influenced by interpersonal and intergroup dynamics based on literature review of CAT. Interviewed patients may have perceived PAFPs, former patients themselves or relatives of former patients, as having similar backgrounds, possibly establishing an in-group bias.28 Conversely, staff interviewers may have been established as an out-group due to intrinsic association with the hospital care team.

Hospital staff play an invaluable role in providing care, and engaging professional staff is an essential component in the success of volunteer programmes.3 Establishing trust and psychological safety among team members is key to creating a team learning environment and guaranteeing strong team performance.29 All members of the healthcare team, including volunteers, need to support the volunteer programme for patients to fully benefit from volunteer services. In fact, having this synergy between volunteers and hospital staff sharing insights, perspectives and approaches can enhance patient care and yield benefits for the services provider.30 We had expected participants would feel more comfortable sharing emotional needs with peers compared with staff, but this difference was not statistically significant. This may be because patients felt equally comfortable with either staff or PAFP interviewers. The hospital staff may have provided psychological safety in the same manner as PAFPs, making patients feel comfortable and empowered to share their emotions.

This research also has compelling applications for the theme ‘Physical Needs’. One in every three US adults has used the internet to research a medical condition which falls under the category ‘Physical Needs’.31 In addition to the internet, patients could communicate with volunteers in order to identify these physical needs as well as get connected with the most appropriate physician to address them. This approach has the advantage of targeted, personal and professional support. Peer to peer healthcare is a field with tremendous potential; currently only one in four patients have reached out to peers with the same illness,31 and we hope that our findings can influence that number to increase.

Lessons and limitations

The SDF programme is ongoing; at this time, the results are mainly applicable to oncology patients such as those included in this study. Interviewers were not matched one-to-one between groups and thus level of involvement was not always congruent. Since data collection was subsequent and not concurrent, PAFPs and staff interviewed different patients. PAFPs and staff conducted interviews in different years; external factors or changes to the hospital system during the study’s timeline could have affected the responses and thus the published results.

The small sample size of this study is another limitation that impedes widespread generalisability. Had there been a larger sample size or a population of patients interviewed from different hospital units, the results of this study may have been different. Additionally, the results are based on responses regarding physicians. Although thematic saturation was achieved, other assessments of healthcare members such as nurses were not explored, which could have provided important information regarding the broader care team.

Different healthcare systems may not be able to allocate the financial or volunteer resources necessary to create an SDF programme. For example, a healthcare system will need to find motivated and qualified individuals who will be dependable volunteers to conduct bedside interviews. For healthcare systems with established volunteer programmes, implementing an SDF programme will require additional volunteer training as well as interview documentation and data storage systems.

In order to support a programme such as SDF, financial and logistic requirements need to be considered, as volunteers will require dedicated hospital administrative personnel to train them, organise their interview schedules and provide authorisation to access hospital facilities. While volunteers are not compensated, acquiring interview documentation and data storage systems can be a financial barrier. Furthermore, all hospital staff need to be aware of the programme’s introduction to keep track of volunteer visits, and to be a source of clinical support if a volunteer has a question or concern when conducting interviews with patients.

Conclusion

This study has uncovered practical insights regarding SDF patients’ assessments of their physicians; these results will be important for improving patient experiences as the USA shifts towards value-based care concomitant with value-based payment models. Our findings show patient and public partners can play an important role in expanding patient–peer interviewer programmes in collaboration with other care professionals. At present, SDF is ongoing and points to the success of the programme. Importantly, we demonstrate that community members such as patients and family are comparable interviewers for needs identification. Having PAFPs on board with staff could increase the time staff would have to focus on meeting medical needs. By eliciting more detailed and personal responses, as well as assessing informational and practical needs, patient–peers are generating transparent and applicable feedback for quality improvement. The results from this study can be viewed by healthcare systems as an additional source of insight for hospital improvement. This research could be expanded to include a cost–benefit analysis and a longitudinal study to assess patient changes over time.