Perceptions of quality of communication in family interactions in neurocritical care.

OBJECTIVE: Given the challenges of patient-provider communication in neurocritical care lacking robust decision-making tools on prognostication, we investigated concordance in perceptions of communication among participants in family discussions and assess the different domains of communication that affect these perceptions.
METHODS: Prospective observational study conducted over 4 months in a tertiary-level academic medical center neurocritical care unit. Our study involved family discussions regarding plan of care for admitted patients observed by a neutral observer. All participants completed a survey. The first four questions rated the understanding of the discussion and general satisfaction; the remaining questions were open-ended to assess the quality of communication by the physician leading the discussion. Responses were scored and compared among participants using a Likert scale. A difference of < 1 in scores among participants was rated as concordance, whereas that of > 1 was designated as discordance. All open-ended responses were classified into six domains.
RESULTS: We observed 35 family discussions. Questions 1 to 3 inquiring on general satisfaction, impact, and understanding of treatment options yielded 99 cross-comparisons per question (297 compared responses). Most responses were either "Strongly Agree" or "Agree," with "Neutral" or "Disagree" responses being more prevalent in Question 2 regarding the impact of the conversation. Overall concordance of responses between participants was 88% with a lower rate of concordance (72%) on Q2. Further open-ended questions queried observers on specific physician-spoken content, and answers were analyzed to identify domains that affected the perception of quality of communication. Education was the most frequently cited domain of communication in response to open-ended questions. Among family and neutral observers, empathy was frequently listed, whereas providers more often listed family engagement.
CONCLUSION: Overall, satisfaction was high among providers, families, and the observer regarding the quality of communication during family discussions in the unit. Perceptual differences emerged over whether this communication impacted healthcare decision-making during that encounter.

INTRODUCTION

Effective communication between healthcare providers and patients/families is essential for patient‐centered care and is pivotal to patient/family decision‐making, particularly in neurocritical care, because the involvement of multispecialty providers can yield inconsistent messages. , , Validated instruments on assessing quality of communication are available but do not inform of perceptual differences among participants in communication encounters. They also do not inform specific domains to help design education strategies to improve communication specific to patient population and provider practices. Our study sought to assess differences in perceptions about communication between providers and patients' families in our neurocritical care unit and assess the importance of different domains of communication to inform future interventions.

METHODS

Participant recruitment and data collection

The study protocol was approved by the local Institutional Review Board. Data were collected in a neurocritical care unit at a tertiary‐level academic medical center. There had been no preceding educational interventions or quality improvement projects targeted towards family communications preceding the study. A neutral observer observed discussions between providers and patients' families. The observer then discussed the study with families or surrogate decision‐makers and obtained informed consent. Each member of the discussion—provider, family, nurse, and observer—completed a questionnaire to evaluate the quality of communication (Table 1).

TABLE 1

Survey questions asked and results of responses to Questions 1 to 3

Survey Questions filled by family members, clinical providers, nurses, and neutral observer
Q1	I am generally satisfied with the provider's communication with family.
Q2	The conversation impacted the family's healthcare decisions.
Q3	The family understands treatment options explained by the provider.
*Q4a	I was comfortable discussing prognosis with family. (Provider)
**Q4b	Overall, I am satisfied with the care my loved one received. (Family)
Q5	What did the physician do or say to influence the family's healthcare decision?
Q6	What did the physician do or say that helped the family understand the treatment options?
Q7	What did the physician do or say that confused the family or was unhelpful?
Q8	What else would you like to tell us about how the physician communicated with the family? What did the physician say that the family liked?
Q9	What did the physician do or say that the family did not like?
Q1 to Q4 Scored on Likert scale (1—strongly agree, 2—agree, 3—neutral, 4—disagree, 5—strongly disagree). Q5 to Q9 Open‐ended answers.

Note: Likert scale 1—strongly agree, 2—agree, 3—neutral, 4—disagree, 5—strongly disagree.

Indicates “presented to provider”.

Indicates “presented to the family”.

We screened clinical encounters between July to August 2015 and June to July 2016 where providers interacted with families to deliver medical information, reviewed plan of care, or addressed goals of care. Two unique neutral observers gather data during this period with one observer in 2015 and another observer in 2016. Both observers were preclinical medical students at similar level of training with no prior healthcare exposure or education healthcare communication. There were no junior new trainees rotating in the ICU or participating in family discussions; hence, this time was selected. Only encounters involving English‐speaking families of patients aged 18 years or older were included. We excluded encounters where providers or families expressed discomfort with the presence of the neutral observer or filling the survey questionnaire (when presented the consent at the end of the encounter), the observer was absent, or families were grieving. We also excluded encounters if a provider refused to complete surveys, or a patient was deceased at the time of communication and the communication primarily centered on patient's demise.

Survey instrument and variables collected

The questionnaire administered was constructed after review of the literature (Table 1). Published validated instruments like Family Satisfaction in the Intensive Care Unit (FS‐ICU) were considered but iterative feedback during trial design emphasized obtaining open‐ended feedback regarding physician practices to inform education on communication. , , , , The request for study participation, consent, and survey were presented to participants at the end of each communication encounter. Family participants were surveyed followed by providers with surveys gathered soon after the encounter to prevent recall bias. Each participant ranked aspects of communication during the encounter on a Likert scale of agreement. Questions 1 to 3 were answered by all participants. Question 4a was answered by providers, and Question 4b was answered only by families. Questions 5 to 9 were open‐ended and used to identify domains of communication that affected participants' perception of quality of communication.

Analysis

Given these data, the standard analysis would use traditional measures of inter‐rater agreement like Krippendorff's alpha, intra‐class coefficients, and Cronbach's alpha but these are not well defined with so little dispersion data as seen in our study. This will be a problem with any summary measure of agreement. We felt that ascertaining the difference between “agree” and “strongly agree” was not important to the discordance as much as difference between “agree” and “disagree”; hence, a three‐point scale made more sense despite available ordinal measures of concordance. For this reason, we defined concordance using an alternative analysis using the degree of difference (DOD) defined (Table 2).

TABLE 2

Pictorial representation of the concordance model created

Concordance = Difference less than equal to 1
1	2	3	4	5
Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
1	2	3	4	5
Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
1	2	3	4	5
Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
1	2	3	4	5
Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree

Note: The highlighted cells are define concordance.

For each encounter, numerical values for each response were compared among respondents (Table 1 and Figure 1). The DOD was calculated to capture concordance; for example, Question 3 states: “The family understands the treatment options explained to them.” If the family's response was “Disagree,” a value of 4 was coded. If the physician's response was “Strongly Agree,” a value of 1 was coded, and the DOD was 3. A DOD ≤ 1 was considered concordance, whereas a DOD ≥ 2 was considered discordance (Table 2).

FIGURE 1

(A) shows the responses to Q1 (“I am generally satisfied with the provider's communication with the family”). (B) shows the responses to Q2 (the conversation impacted the family's healthcare decisions). (C) shows the responses to Q3 (the family understands treatment options explained by the provider)

Each pair of responses for each of the first three questions was coded as concordant or discordant. A repeated measures logistic regression model was fit with terms for the subject, the pair of responders (eg, family and provider, nurse, and observer), and the question, allowing a comparison of the rates of concordance between the questions, and between the pairs of responders, while adjusting for the correlation among responses.

In addition, data from Questions 1 to 3 were reanalyzed using a three‐point scale where “Strongly Agree” and “Agree” were classified as “Agree”; “Neutral” remained the same; and “Disagree” and “Strongly Disagree” were classified as “Disagree.”

Table 3A presents data using the original five‐point Likert scale, whereas Table 3B presents data using a condensed three‐point scale:

TABLE 3

Rater ratings using (A) the five‐point scale (B) the three‐point scale

(A)
Question	Rater	Response
		Strongly Agree		Agree		Neutral		Disagree		Strongly Disagree
		N	Percent	N	Percent	N	Percent	N	Percent	N	Percent
I am generally satisfied with provider's communication with family	Nurse	6	54%	4	36%	0	0	1	9%	0	0
	MD	12	54%	10	45%	0	0	0	0	0	0
	Family	12	54%	10	45%	0	0	0	0	0	0
	Neutral	18	81%	4	18%	0	0	0	0	0	0
The conversation impacted the family's healthcare decisions	Nurse	4	36%	4	36%	3	27%	0	0	0	0
	MD	9	40%	5	22%	8	36%	0	0	0	0
	Family	9	40%	6	27%	4	18%	2	9%	1	4%
	Neutral	11	50%	6	27%	5	22%	0	0	0	0
The family understands treatment options explained by the provider	Nurse	5	45%	5	45%	0	0	1	9%	0	0
	MD	16	72%	6	27%	0	0	0	0	0	0
	Family	15	68%	6	27%	1	4%	0	0	0	0
	Neutral	12	54%	8	36%	1	4%	1	4%	0	0

Note: Krippendorf's alpha, calculated on the original data (five‐point scale), because it is said to perform better using the five‐point scale, shows poor to moderate agreement beyond chance. The discrepancy between high percent agreement and low Krippendorf's alpha is due to the fact that the method performs best when there is more spread of responses across ratings—with this data, most ratings occur as agree/highly agree. For this reason, it is not clear that presenting Krippendorf's alpha (or any other measure of agreement like this) will be helpful to the reader. This will be a problem with any summary measure of agreement (Cronbach's alpha, Intra class coefficient, etc). Obviously when the percent agreement is 90%+, it does not make sense for Krippendorf's alpha to be low—it “should” be much further from 0.

Finally, responses to open‐ended questions were divided into six domains, initially using the Bayer Institute for Health Care Communication E4 Model—Education, Empathy, Engagement, and Enlistment. Each comment was analyzed by a blinded scorer using these domains. Post‐hoc analysis revealed several comments addressing Speech Mechanics and Settings; these domains were added, resulting in six domains for the analysis. (Figure 2).

FIGURE 2

(A) shows the total percentage of open‐ended remarks made in each domain. (B) shows the percentage of remarks made in each domain categorized by participant group. (C) shows the descriptive remarks by participants in survey with examples of domain assignment

RESULTS

We examined 35 encounters; we excluded 13 encounters based on our criteria. Collectively, 77 surveys were completed: 22 by providers (18 led by a neurointensivist care physician, four were led by a resident or an advance practice provider), 22 by the observer, 22 by families, and 11 by nurses. Mean values are reported in Table 1, and responses in each category are shown in Figure 1 and Table 3.

Only 28 of 231 responses to the first three questions (12%) were Neutral, Disagree, or Strongly Disagree. For this reason, we defined concordance using the DOD defined above (Table 2). Analysis of Questions 1 to 3 yielded 99 cross‐comparisons per question and a total of 297 responses. Two hundred and sixty‐one responses demonstrated concordance and 36 showed discordance (88% concordance, 12% discordance). Most responses for Questions 1 and 3 were in the “Strongly Agree” or “Agree” category. Concordance between all groups was high, particularly for Questions 1 and 3 (96% and 94%, respectively), suggesting that all parties agreed on quality of the communication. Question 2 on whether communication impacted healthcare decisions had a lower rate of overall concordance (72%) than the other two questions (P < .01). This discordance was greatest when comparing family/provider and family/nurse responses, although there were no statistically significant differences between pairs of responders. Analysis of exact concordance on the condensed three‐point scare yielded similar results (Table 4) validating our approach. For Question 1, with 1 exception, all raters agreed with the content of the statement and therefore with each other. For question 2, there is more variability in ratings, with MD's appearing less likely than other raters to agree with the statement (63% vs 68% to 77% for other raters). For Question 3, with the exception of four ratings, all raters agreed with the statement, also indicating there would be a high level of between‐rater agreement.

TABLE 4

Analysis of inter‐rater agreements

Question	Rater #1	Rater #2	# Rating Pairs	# Pairs a Agreeing	Percent Agreement	Krippendorf's Alpha (95% CI) b
I am generally satisfied with provider's communication with family	Family	MD	22	22	100%	b
	Family	Neutral	22	22	100%	b
	MD	Neutral	22	22	100%	b
	Nurse	Family	11	10	90%	b
	Nurse	MD	11	10	90%	b
	Nurse	Neutral	11	10	90%	b
The conversation impacted the family's healthcare decisions	Family	MD	22	10	45%	0.0 [0, 0.34]
	Family	Neutral	22	15	68%	0.36 [0.0, 0.66]
	MD	Neutral	22	13	59%	0.0 [0.0, 0.37]
	Nurse	Family	11	7	63%	0.29 [0.0,0.62]
	Nurse	MD	11	7	63%	0.28 [0.0, 0.76]
	Nurse	Neutral	11	7	63%	0.0 [0.0, 0.59]
The family understands treatment options explained by the provider	Family	MD	22	21	95%	0.08 [0.0, 0.48]
	Family	Neutral	22	19	86%	0.41 [0.04, 0.72]
	MD	Neutral	22	20	90%	0.22 [00, 0.65]
	Nurse	Family	11	9	81%	0.0 [0.0, 0.54]
	Nurse	MD	11	10	90%	0.36 [0.0, 0.84]
	Nurse	Neutral	11	10	90%	0.50 [0.0, 0.92]

Agreement when both raters provide the same rating (eg, both say agree/strongly agree, neutral, or disagree/strongly disagree).

Krippendorff's alpha calculated on the full five‐point Likert scale; not well defined when little to no disagreement between raters. Lower end of confidence interval truncated at zero with lower bound less than zero.

The discordance among participants for Question 1 did not affect respondents' general satisfaction with provider's communication during the encounter (Table 3). For encounters with complete concordance, 73% of respondents strongly agreed that they were satisfied with the provider's communication. Among encounters with at least one discordant crossmatch, that number was 70%. Even for encounters where three or more cross‐matched responses were discordant, participants answered “Strongly Agree” for Question 1 regarding satisfaction with the encounter 68% of the time. Table 4 presents the between‐rater agreement by question using data on the three‐point scale. Between‐rater percent agreement is extremely high (90%‐100%) for Question 1. Between‐rater percent agreement for Question 2 was more modest (ranging from 45% to 68%). High percent agreement between raters was also seen for Question 3 (81%‐95%).

Question 4a assessed the provider's comfort with discussing the patient's prognosis with the family. All 22 providers reported that they strongly agreed or agreed (n = 16 and 6, respectively) that they were comfortable discussing the prognosis with families. Question 4b assessed the family's general satisfaction with the care the patient received. All respondents either strongly agreed or agreed (n = 19 and 3, respectively) that they were satisfied with the quality of care.

Questions 5 to 9 yielded 148 responses; six responses were recorded as none or not sure. Some respondents provided multiple remarks (159 positive, 17 negative), yielding 176 keywords classified into six domains. Common remarks involved educational content (46%), empathy (21%), and engagement (19%) (Figure 2). The observer, families, and nurses primarily focused on the educational content and empathy (86%, 84%, and 73%, respectively). Providers commented on either educational content or empathy in 49% of remarks. Providers and the observer noted the “physical setting of the meeting” in 14% of remarks. Empathy was the most frequently cited domain by families (31%) and the observer (30%).

DISCUSSION

Our project shows a survey‐based approach to investigate concordance in perceptions of communication among participants in family discussions and identify domains of communication in a neurocritical care setting. Most participants in our cohort felt that treatment options were adequately conveyed and were satisfied with the communication, although the impact on healthcare decisions was reported lower than expected.

There are several factors that influence perceptions during communication in the critical care unit. Some of these factors include the frequency of physician communication, inclusion, and support during the interactions and feeling of control over decisions around care of the loved ones. Several barriers have been identified affecting perceptions of communication with family and impact of this communication on healthcare decisions. One study identified more than 30 barriers spanning four domains ranging from logistics, clinician discomfort, inadequate training, and fear of conflict. Patients or their families may have preexisting religious beliefs or prior experiences influencing trust (or mistrust) in the healthcare system. Prior plan of care discussions with different providers may have created inconsistent messaging regarding plan of care discussions that may have influenced the decisions already. Some providers themselves may feel stressed or poorly prepared leading these discussions due to their personal beliefs, prior experiences, or lack of training in leading conversations in critical care settings where morbidity is high and prognostication is uncertain.

Our study showed relatively high performance and concordance compared to published literature likely due to selection bias as most conversations were led by fellowship‐trained neurocritical care physicians who had significant exposure to such discussions during their training. Our unit‐based governance also has a significant emphasis on family experience that may have influenced an overall favorable environment.

Education was the most frequently mentioned domain of communication in open‐ended questions among all groups. Engagement and enlistment were not frequently mentioned by families. Remarks from families and the observer noted empathy, which was notably absent in provider remarks. These findings led us to create an education module implemented as a part of our curriculum highlighting ways to improve communication. Given the extremely positive results of the pre‐module surveys, post‐module assessment was not felt to provide contributory information to boost initiatives to improve communication.

Our limitations included small sample size, restriction to English‐speaking participants, inability to rule out selection bias or Hawthorne effect among respondents and high degree of competency presumed from attending physicians directing most conversations. It is possible that families with discomfort or conflict during these discussions refused consent for the study creating self‐selection of satisfied respondents. Furthermore, conversations in family lounge areas mentioning the study may have affected expectations of family satisfaction and concordance.

This study did not assess other factors that influence patient decision‐making (eg, family religiosity, educational level of decision‐makers, socioeconomic status). Such factors may underlie discordant provider and family perceptions of how provider communications influenced family decision‐making.

Traditional summary measures of agreement like Krippendorf's or Cronbach's alpha, and intra‐class coefficients were also considered. Krippendorf's alpha calculated on the original data showed poor to moderate agreement beyond chance; the discrepancy between a high percent agreement and low Krippendorf's alpha may indicate that the method performs best with responses that are more diverse.

Overall, our study showed high satisfaction among providers, families, and the observer regarding quality of communication during family discussions in the neurocritical care unit.

Our study identified the variability in perceptions among various domains of communication. This was useful for designing and testing educational interventions involving family communication.

Recommendation

The results of the survey informed an education module implemented as a part of the neurocritical care curriculum for trainees and advanced practice providers rotating in the neurocritical care unit. This curriculum was included in the orientation of all new providers and included in the neurocritical care handbook (please see Appendix S1).

FUNDING

This study was funded by Medical Student Research Program supported in part by T35DK007400 training grant, which provided support for medical students to participate in the study through the summer.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

AUTHOR CONTRIBUTION

Conceptualization: Russell Stewart, Kyle Hobbs, Kristopher Dixon, Roberto Alejandro Navarrete, Jannat Khan, Mary Petrulis Wren, Mollie Canzona, Aarti Sarwal

Data Curation: Kristopher Dixon, Roberto Alejandro Navarrete, Jannat Khan, Mary Petrulis Wren, Mollie Canzona, Aarti Sarwal

Formal Analysis: Kristopher Dixon, Roberto Alejandro Navarrete, Jannat Khan, Mollie Canzona, Aarti Sarwal

Investigation: Jannat Khan, Aarti Sarwal

Methodology: Russell Stewart, Kristopher Dixon, Roberto Alejandro Navarrete, Jannat Khan, Mary Petrulis Wren, Mollie Canzona, Aarti Sarwal

Project Administration: Russell Stewart, Kyle Hobbs, Kristopher Dixon, Roberto Alejandro Navarrete, Jannat Khan, Mary Petrulis Wren, Mollie Canzona, Aarti Sarwal

Resources: Roberto Alejandro Navarrete, Aarti Sarwal

Supervision: Aarti Sarwal

Validation: Aarti Sarwal

Visualization: Russell Stewart, Aarti Sarwal

Writing Original Draft Preparation: Russell Stewart, Kyle Hobbs, Kristopher Dixon, Roberto Alejandro Navarrete, Jannat Khan, Mary Petrulis Wren, Mollie Canzona, Aarti Sarwal

Writing Review & Editing: Russell Stewart, Kyle Hobbs, Kristopher Dixon, Roberto Alejandro Navarrete, Jannat Khan, Mary Petrulis Wren, Mollie Canzona, Aarti Sarwal

All authors provided final approval of the version to be published.

Aarti Sarwal supervised the project and research team members and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

TRANSPARENCY STATEMENT

The manuscript is an honest, accurate, and transparent account of the study being reported; no important aspects of the study have been omitted; and any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Appendix S1. Supporting Information.

Click here for additional data file.