Protocol for a mixed methods study of hospital readmissions: sensemaking in Veterans Health Administration healthcare system in the USA.

INTRODUCTION: Effective delivery of healthcare in complex systems requires managing interdependencies between professions and organisational units. Reducing 30-day hospital readmissions may be one of the most complex tasks that a healthcare system can undertake. We propose that these less than optimal outcomes are related to difficulties managing the complex interdependencies among organisational units and to a lack of effective sensemaking among individuals and organisational units regarding how best to coordinate patient needs. METHODS AND ANALYSIS: This is a mixed method, multistepped study. We will conduct in-depth qualitative organisational case studies in 10 Veterans Health Administration facilities (6 with improving and 4 with worsening readmission rates), focusing on relationships, sensemaking and improvisation around care transition processes intended to reduce early readmissions. Data will be gathered through multiple methods (eg, chart reviews, surveys, interviews, observations) and analysed using analytic memos, qualitative coding and statistical analyses. We will construct an agent-based model based on those results to explore the influence of sensemaking and specific care transition processes on early readmissions. ETHICS AND DISSEMINATION: Ethical approval has been obtained through the Institutional Review Board of the University of Texas Health Science Center at San Antonio (approval number: 14-258 hour). We will disseminate our findings in manuscripts in peer-reviewed journals, professional conferences and through short reports back to participating entities and stakeholders. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Using Eisenhardt’s recommendations for building theory from case studies, this study samples 10 sites with a minimum of 2000 discharges per year, all of which have attempted efforts to improve hospital-to-home care transition processes and have either worsening or improving hospital readmission rates over a 5-year period, allowing us to explore organisational characteristics leading to these performance patterns.

For each site, we create an in-depth qualitative organisational case study of relationships, sensemaking and improvisation around care transition processes, from which we will build an agent-based model to explore how system elements may impact hospital readmission rates and identify potential leverage points for new types of interventions.

Limitations include the single point in time data collection, all facilities are drawn from a single healthcare system (the Veterans Health Administration) and the study is observational rather than interventional.

Introduction

Complex systems cannot be understood by breaking their processes down into component parts or into individuals’ jobs, even though this is often our first response to solving complicated problems in healthcare.1 2 Effective healthcare delivery requires effective management of interdependencies between socially distinct professions and between organisational units with unique perceived purposes and purviews. Within well-integrated systems, patients navigating unit boundaries should feel like system components form a continuum that communicate and cooperate for the explicit purpose of patient wellness.

As the largest integrated healthcare system of the USA, the Veterans Health Administration (VHA) is theoretically positioned to deliver integrated care along such a continuum. Despite this, VHA’s performance has been similar or worse than Medicare providers with regard to outcomes reflecting complex interdependencies, such as unplanned hospital readmissions.3 We propose that these less than optimal outcomes are related to difficulties managing the complex interdependencies among VHA organisational units and to a lack of effective sensemaking among individuals and organisational units regarding how best to coordinate Veteran needs.

Early readmissions as a persistent problem

Hospital readmissions continue to receive significant attention as a source of potential waste and a marker of poor quality. Although the policy emphasis on readmissions is recent,4 early readmissions have been proposed as a quality indicator for at least 22 years.5 Numerous studies assessing the extent of preventability of early readmissions have had widely varying estimates: 5%–79%.6–8

Readmission rates have been declining but are still felt to be unacceptable.9 VHA hospital-wide risk-adjusted 30-day readmission rates gradually dropped 3% from 1997 to 2010 (16.5%–13.8%),10 and have remained around 13% (IPEC readmission cube on VHA Support Service Center, accessed 19 May 2017).

Why has reducing early hospital readmissions been such a persistent challenge? Reducing readmissions within 30 days may be one of the most complex tasks in a healthcare system. First, success depends on the intersection, coordination and collaboration of many parts of the system Second, patients and their caregivers are in control of many of the factors that will determine their ability to stay out of the hospital, and healthcare delivery systems may not recognise the challenges faced postdischarge. Third, with a focus on shortening hospital length of stay, assumptions have been made about who is responsible for different aspects of care, with gaps occurring when expectations are not congruent. Fourth, there is a dearth of geriatricians who might have more insight into frail patients’ needs and be better equipped to deal with the large numbers of chronically ill elderly.11 Fifth, due to ongoing fragmentation of provider-patient relationships, there may be both a lack of recognition of and communication regarding the need for palliative care. Finally, technologies and processes that prolong life may require a greater number of appropriate hospital admissions over an individual’s life course.

Given the complexity of understanding all elements contributing to readmissions, it is no surprise that preventing early readmissions remains a challenging healthcare issue.

Risk prediction models for readmissions

One approach to reduce readmission rates has been to implement risk prediction models to identify and target interventions towards those most at risk for early readmission. Kansagara et al12 reviewed 26 unique models. They concluded that most readmission risk prediction models performed poorly and as yet are not useful in clinical settings. This finding was corroborated by a systematic review by Zhou et al,13 which found that while risk prediction models are growing in number and condition specificity, they show only moderate discriminative ability. These models typically focused on risk characteristics of the patients and not characteristics of institutional behaviour that might put patients at risk.

Care transitions studies

Another approach to reducing readmission rates is through care transition interventions. Hansen et al14 found that of 16 randomised, controlled trials of interventions to improve 30-day rehospitalisation rates, only 5 documented statistically significant improvement in reducing rehospitalisations. Four of these five tested multicomponent discharge bundles, however 11 other randomised controlled trials, some of which also used bundles with similar elements, failed to show improvements. Leppin et al15 found the majority of reviewed trials (38 of 42) did not have a significant effect on readmissions; however, studies with five or more unique activities in the intervention were more effective at reducing readmissions than those with two or more activities. One interpretation of these mixed findings from the perspective of complexity science is that interventions focus on breaking down processes into component parts or on changing the behaviours of individuals (assigning specific individuals to specific tasks) but do not address the interdependencies and boundary crossings that make the transitions so difficult.

Despite the ambiguity of the evidence and because of the burden of readmission for both the patient and the system, many individual VHA facilities are trying some of the more promising of the above models (eg, Project RED, Project BOOST). There have also been VHA-sponsored efforts, such as to address chronic heart failure readmissions16 and to enact transition management initiatives, and nationwide policies to conduct discharge medication reconciliation and to conduct postdischarge follow-up calls. However, there are few care transition elements mandated to be implemented across VHA facilities.

Complexity science as a theoretical lens for understanding why reducing readmissions is so difficult

The application of complexity science to healthcare systems can provide new insights to the issue of readmissions. Defining characteristics of complex adaptive systems are diverse learning agents who interact non-linearly and who self-organise. These complex systems co-evolve with their environment and have emergent properties that are not predictable. Due to the systems’ non-linearity, inputs and outputs are not necessarily proportional.17 Even though organisations might implement care transition programmes, the amount of effort put into their programmes is not necessarily proportional to readmission rate outcomes.

The inherent non-linearity of complex systems also leads to uncertainty. This may be particularly true during transitional periods for patients, when patients’ recovery is not yet assured, the home environment is often not well known to the staff and the possibility of developing a relapse is significant. In these situations, uncertainty is compounded.18 19 Implementing new initiatives and changing processes also introduce uncertainty. An implication of this is that improvement efforts need to focus on process of care, and on the relationships between and interdependencies among healthcare providers.1 2 20

Relationships, sensemaking and improvising

Relationships among healthcare workers are the foundation for the social activities that occur during patient care, like transitions of care. The framework of work relationships by Lanham et al proposes that seven characteristics define effective relationships in healthcare: trust, mindfulness, heedfulness, respectful interaction, diversity, social and task relatedness and rich and lean conversation.21 These characteristics interact with how individuals and groups of providers reflect, make sense and learn in ways that shape the quality of patient outcomes. Through relationship infrastructure, care transitions staff can coordinate as an effective, interdependent group in patient care. However, fostering relationships to improve care delivery is not something to which healthcare organisations have traditionally paid attention, even though data speak to its importance.21–23

Differences in relationship infrastructures across services, teams and organisations may help explain the varying impacts of care transition interventions. The relationship infrastructure can give way to activities, such as sensemaking and improvising, which help providers and other organisational staff manage uncertainties and stressors. In sensemaking, people assimilate information, reach conclusions and take steps to act.24 In the inpatient setting, sensemaking can occur in relation to individual patient diagnosis and care, as well as understanding more broadly patient illness trajectories and how their condition changes over time.25

Preventing early readmissions via sensemaking involves multiple sets of individuals interacting to make sense beyond the physician team. Our model below summarises these interdependencies (figure 1). The trajectory of the patient’s illness needs to be understood as it continues in the home or next institutional environment and in relation to how well the home environment meets patient posthospitalisation needs, what actual supports need to brought together, the level of understanding of the patient and/or caregiver of the self-management that will need to occur, understanding of funding mechanisms and more. While checklists provide reminders of what needs to be done, they do not necessarily help providers make sense of what needs to be done for whom, or when or how to engage others to help.

Figure 1

Model of care transitions.

Improvising is varying what one does based on the context and situation at hand.26 27 Physicians describe the importance of improvisation amid new or uncertain situations in patient care.27 Thus, improving care transitions teams’ ability to improvise may be a powerful strategy for targeting activities to the needs of individual patients and decreasing readmissions.

Project aim

We are studying care transition interventions aimed at reducing early readmissions as an exemplar of processes requiring a high level of interdependencies and sensemaking. By investigating VHA facility cases that have attempted interventions to improve care transitions and have had either improvement or worsening in their readmission rates, we will improve our understanding of the care transition processes themselves and the sensemaking within the organisation needed to implement change when there is no single part of the organisation responsible for the outcome.

Objective 1: conduct in-depth qualitative, organisational case studies to explore relationships, sensemaking and improvisation in six facilities with improving and four facilities with worsening early readmissions rates between fiscal years 2006 and 2011, all of which engaged in care transition interventions to improve early readmissions.

Objective 2: extend learning from and enhance generalisability of the case studies, using agent-based model (ABM) to simulate facilities implementing care transition innovations and to explore both specific care transition processes and elements of sensemaking as they prevent early readmissions, or not, as possible system outcomes.

Methods and analysis

Study design overview

We are conducting a mixed method, multistepped study using concurrent triangulation. It will be conducted in two parts: the first part will be an in-depth qualitative organisational case study; the second part will be constructing an ABM based on those results.

Objective 1. Organisational case studies

Case sample and individual recruitment within cases

Given that the intent of the study is to build or extend theory, not to test existing theory, we are using Eisenhardt’s recommendations with regard to sampling for case studies in her methodological review, ‘Building theories from case study research’.28 In this context, cases are chosen on theoretical grounds and not for statistical reasons. Cases may be chosen to replicate previous cases or extend emergent theory or they may be chosen to fill theoretical categories and provide examples of polar types, in which the process of interest is ‘transparently observable’.28 29 Random selection is neither necessary nor even preferable. The goal of the theoretical sampling is to choose cases which are likely to replicate or extend the emergent theory. In this spirit, our criteria for case selection concerned facility size, trending 5-year readmission rates and documented care transition improvement efforts (table 1).

Table 1

Case study eligibility criteria

Eligibility criteria	Process for establishing eligibility
Criteria 1. A minimum of 2000 admissions per year to the facility	After visually reviewing the all-cause medical surgical readmission rates for 2006–2011 for all VHA hospitals and comparing facilities with varying admission totals, we identified that facilities with >2000 admissions/year had less dramatic variability in their year-to-year readmissions rates. We also felt that facilities with larger numbers of admissions were more likely to spend intellectual and human resources on care transitions.
Criteria 2. Significantly increasing or decreasing all-cause medical surgical readmission rate between fiscal years 2006 and 2011	Using the unadjusted readmission rates obtained from the IPEC readmission cube,46 we tested whether the change in rate over 5 years was significant or not. Eleven facilities were improvers (declining readmission rates), nine facilities had significantly worsening rates (increasing readmission rates) over that time. We chose facilities with significantly changing rates as we wanted to explore attempts at innovations and changes in the outcomes of interest to the facility.
Criteria 3. Two or more care transition innovations identified	Within the two different readmission performance groups (improving or worsening), we narrowed selection further using multiple sources of data regarding care transitions innovations within the VHA including a national survey of Utilization Management Nurses conducted in 2013, listings of all transitional care pilot projects funded by a VHA initiative called the Geriatrics T21 funds, and listings of all VHA Flow Improvement collaboratives on care transitions in the same time frame. We felt documented efforts to improve care transition processes provided evidence of some attempts at bettering readmission rates but did not expect that these would be the only care transition or rate improvement efforts undertaken by the sites. By comparing each of these sources for information, we identified 13 facilities, meeting the above criteria, with evidence of two or more innovations taking place around care transitions and prevention of readmissions. We eliminated from the potential sample pool the seven facilities for which we did not have evidence of two or more care transitions innovations.

VHA, Veterans Health Administration.

Within each facility case, individuals will be recruited to participate in interviews, focus groups, observations and/or surveys using purposive sampling.30 Purposive sampling allows us to identify and recruit individuals with specific experiences and knowledge that will inform our case building. We will use information from facility websites (eg, organisational charts, service rosters) and the VA’s Microsoft Outlook contact list to identify individuals occupying specific roles. During site visits, snowball and convenience sampling will also be used to identify people with knowledge of site care transition innovations and experience with care transition practices.

Potential participants will be invited to participate through email and/or face-to-face. Specific forms of sampling and recruitment will vary based on data collection activity (table 2). Note, recruitment for one activity does not preclude recruitment for other activities. For example, a hospitalist might be engaged in an interview as well as an observation of her medicine rounds. At each site, investigators will aim to balance recruiting to obtain diverse, representative perspectives and to generate deeper knowledge about specific experiences.

Table 2

Participant recruitment for each case study site

Activity	Population	Description of recruitment
Interviews	Service leaders (n=~10)	Individuals from medicine, nursing, social work, pharmacy and primary care leadership (ie, service chiefs and supervisors) will be identified through organisational charts available on facility websites or sharepoints, the VHA outlook contact list or by other staff at the facility. They will be contacted by phone or by email to participate in interviews.
Chart reviews	Patients (n=10)	Project staff and investigators will review the charts of a random selection of 10 veterans admitted to the facility’s hospital within the 3–6 months before the scheduled site visit. Five of the veterans will have had 30-day readmissions following their index admissions and five of them will have not. All 10 veterans must meet the following inclusion criteria at the time of the index admission: (a) inpatient or outpatient contact in the previous year with a VHA provider; (b) a Charlson Comorbidity Index47 of two or more; (c) discharge from a general medicine unit at the case study hospital within the sampling period; (d) discharge diagnosis of chronic obstructive pulmonary disease, chronic heart failure and/or pneumonia and (e) discharge to home. Patients are excluded if they are discharged to a long-term care or skilled facility. For each site, a VA data analyst will provide the team with a sample of the first 10 readmitted and 10 non-readmitted patients meeting these criteria. The project coordinator will verify that these patients meet eligibility criteria and assign the first five in each group which meet eligibility criteria to be reviewed. A waiver of consent was obtained for the sample of patients for whom we conduct chart reviews.
Interviews	Frontline providers (n=15–20)	We will sample one to four providers from each of the following roles: hospitalists, inpatient medicine nurses, inpatient social workers, pharmacists who deal with discharge education and supply of medications to patients on discharge, primary care team providers and, when present, dedicated care transition staff (eg, patient care coordinators). Depending on each site’s processes and programmes, interviews may also be held with representative staff from palliative care, subspecialty care (eg, geriatrics, cardiology), telecare, utilisation management and others as appropriate.
Focus groups	Frontline providers (n=1–2)	One to two focus groups, comprised 4–10 individuals, will be held at each site. For each focus group, the team will aim to recruit one to two staff to represent the following roles: hospitalists, nurses, social workers, pharmacists and any roles important to care transitions at that site (eg, patient care coordinators, utilisation management nurses). Investigators will recruit frontline staff using snowball and quota sampling methods.
Observations	Frontline providers (n=17–30)	Staff participating in discharge planning, performing care transition tasks (eg, discharge education) and doing day-to-day work on medicine units (eg, rounds) will be eligible for observation. Investigators will purposively recruit participants for observations before the site visit (eg, through email) and face-to-face during the site visit prior to the start of observations. The specific types of activities observed and number of times they are observed will vary depending on the facility, but the team will broadly aim to observe three to six medicine rounds, three to six discharge planning meetings, four med-surg unit observations, three to six job role shadowing and four to eight patient discharge educations. Observation lengths will also vary, from 10 min (eg, patient discharge education) to 3 hours (eg, medicine rounds). During observations, as necessary, researchers will identify themselves to obtain verbal consent from other patients, staff and other individuals that enter the field of observation once it has commenced. Investigators will use discretion to cease observations if they determine an individual may not be in a position to provide informed consent (eg, a critically ill patient). Data collection will cease if any person declines to be observed.
Surveys	Frontline providers (n=15)	Members of the inpatient care transition teams (eg, hospitalists, social workers, nurses, pharmacists) and any frontline staff members with a direct role in care transitions (eg, primary care nurses and physicians) will be invited to participate in an anonymous survey. They will be identified during data collection activities (eg, observing discharge planning meetings, individual interviews), and invited to participate either by email or in person. Everyone encountered who is eligible to participate will be recruited. Surveys can be filled out online (through REDCap) or by handing in a paper copy, neither form collects identifying information and investigators will not make any notes about who turns in paper forms of the survey.
Interviews	Patients (n=5)	Five patients being discharged from medicine units to home will be recruited for interviews. Patients will be sampled using convenience methods and identified by frontline staff.
Exit debrief	Facility leaders (n=2–8)	During early email communications with site representatives, facility leadership will be asked to attend an hour long exit debrief on the last day of the team’s site visit. Facility directors and chiefs of staff will be invited, along with anyone else they deem appropriate.

All providers and staff recruited to participate in interviews, focus groups, observations and surveys will be consented using a verbal consent form distributed through email and/or in hard copy form. The verbal consent form outlines the purpose of the study and that participation is voluntary. Investigators trained in subject recruitment will ensure the potential participants read and understand the form, and agree to participation before engaging subjects in research. A waiver for the documentation of signed consent was obtained as a further level of protecting VHA staff participants’ anonymity. Patients will be consented through a signed consent process and asked to sign a Health Insurance Portability Accountability Act form (a form required by the US law to protect personal health information and medical records) to allow researchers to access their electronic health record. If at any point a potential or consented participant expresses a desire to not participate, investigators will discontinue recruitment or data collection efforts with them.

Data collection

We will gather and organise preliminary data before the site visit to delimit the organisational context and identify particularly promising areas for interviews and observations. We will visit each facility for a 5-day on-site visit. We will do follow-up data collection, when necessary by phone and protected correspondence. We will undertake to complete roughly one site visit per quarter with 2–2.5 months of qualitative data analysis between. Due to the planning for the ABM (see below), we anticipate that parameters and agent characteristics that we learn about in early interviews will suggest questions and observations for subsequent site visits, checking for the presence or absence of these parameters or agent characteristics. Specific time frames and methods used will be responsive to local context and what we learn during previous site visits.

Team investigators hold advanced degrees in a diversity of fields, including medicine (JP, LL), anthropology (EF, LP), psychology (PN) and business (HL, LL). They each have at least 10 years of experience conducting qualitative research. If not already experienced with complexity theory and ABM, each was provided orientation to these approaches before the study commenced.

Case data collection

Each site visit will follow the same general data collection approach, with site-specific variations depending on local context (eg, care transition processes, staffing and roles) (table 3). Preparation will involve logistical activities and data gathering through leadership interviews and chart reviews. The 5-day site visit will include a continuation of activities started before the site visits, as well as additional interviews, observations of care transition work, focus groups and staff surveys. Follow-up patient interviews will occur about a month after the site visit.

Table 3

General schedule for case study data collection and analysis for each site

<-----------------------------------------------------------3 Months----------------------------------------------------------->
	Presite visit	5-Day site visit	Postsite visit
Data collection	Facility background Chart reviews Leadership interviews	Leadership interviews (cont.) Frontline provider interviews Patient interviews Focus groups Observations Frontline provider surveys Care transition process checklist	30-Day postdischarge Interviews with patients
Data analysis	Chart review memos	Observation scoring Team debrief memos	Facility reflection Qualitative analysis in NVivo quantitative analysis

Throughout the course of case study data collection, team members will talk about what they are finding and fine-tune questions and approaches so that data collection is responsive to site processes and contexts. Decision-making during weekly meetings will be documented in detailed meeting notes. Changes in data collection will be recorded in site-specific data protocol.

Each site visit will be made by three investigators trained and experienced in qualitative methods (JP, PN, LP and/or HL). Investigators have no relationship with participants prior to the start of the study. Data collection instruments will be tested at the investigators’ home facility to ensure inter-rater reliability.

For each case study, qualitative and quantitative data will be collected in the form of background documents, patient chart reviews, semi-structured interviews, focus groups, observations, checklists, debriefments and surveys (table 4).

Table 4

Case study data collection

Type	Description	Purpose and link to aims
Facility background	The project coordinator and investigators conducting the site visit will begin to compile background information on the facility as soon as a visit date is set. Sources of information will include VHA Support Service Center for performance metrics (eg, 30-day risk standardised readmission rate) and the facility webpage and sharepoint (eg, for unit structure, inpatient discharge policies, care transition-related pilots). Investigators will also add information about site-specific roles, care transition processes (eg, discharge planning) and readmission-reduction efforts gathered during presite visit interviews (see below).	Facility background documents will inform site visit planning and data gathering activities, and serve as broader context for the case study.
Patient chart reviews	Project staff and investigators performing chart reviews will be assigned two to three patients to perform chart reviews through the electronic health record on the VHA’s Compensation and Pension Record Interchange. The following chart note types will be reviewed for each hospitalisation: medicine history and physical, nursing admission, social work screening/assessment, interdisciplinary treatment team plan, nursing discharge, social work discharge, pharmacy discharge, medicine discharge, discharge summary, postdischarge primary care nurse follow-up call and any site-specific care transition notes. Chart reviews involve two steps and use structured forms in REDCap48: 1. Chart note type review: for each index admission and readmission, reviewers identify and review two to three instances of the note types of interest (see above). Structured reviews occur through a REDCap form. Each note is assessed for whether they contain (a) documentation of widely agreed on readmission risk factors and (b) cosigners. 2. Patient case study: for each patient, reviewers will read additional notes to type a brief, de-identified case study narrative of the patient’s course during and after the admission(s). Reviewers will use an additional structured REDCap form to document patient-specific readmission risk factors and characteristics (eg, non-VHA insurance coverage). The case study narrative will also be copied into this form.	Recently discharged patients’ chart notes will be reviewed for two primary purposes1: to identify if, where and how sites’ systematically capture and communicate information about widely agreed on readmission risk factors and2 to synthesise information gleaned through specific patient case reviews to create individual case profiles. The latter will describe, for example, the documentation of index admission regarding what plans were in place, how robust were the plans, how well did they consider issues likely to arise, what issues did arise, and for the readmissions, cause of readmission and preventability.6 7 49 This information will inform our understanding of organisational relationships (eg, who is communicating) and sensemaking (eg, what information is available for sensemaking about risk for readmissions).
Service leader interviews	Service leaders will participate in interviews using a guide that collects basic information about service composition and processes, as well as middle-level supervisors to contact about frontline recruitment. Leaders involved in efforts to reduce hospital readmissions at the facility or who are knowledgeable about facility care transition practices, will be invited to answer additional questions about historical and current care transition processes at their facility (see online additional file 1). Interviews generally will occur by phone or Microsoft Lync or Skype for business. Interviews with leadership that do not take place before the site visit, will occur on site in a private setting of the participants’ choosing. The interviews will last between 10 and 30 min. When possible, interviews will be audio recorded and transcribed; written notes will be taken and typed up when audio recordings are not available.	These interactions will serve to (a) inform service leadership of the project and ensure their support of the participation of their service staff and (b) identify the best ways to recruit staff for interviews and focus groups, and observe care transitions. These interviews will also inform our understanding of organisational relationships and processes.
Frontline provider interviews	Semi-structured interview guides will cover the history of care transitions at the facility, what motivated and who was involved in those changes, sensemaking around specific patient cases, and current care transitions processes and support at the facility (see online additional file 1). Interviews will last between 20 min and an hour. Interviews will take place in private spaces within the facility and be audio recorded. Audio recordings will be transcribed.	Frontline provider interviews will provide information about organisational processes, relationships and sensemaking.
Focus groups	One to two, interdisciplinary focus groups will be held at each site. Staff will be purposively sampled so that focus groups have representatives from the services of interest. One investigator will facilitate the focus group, while at least one investigator assists. The investigators will follow a focus group script (see online additional file 1) that probes into care transition processes, sensemaking around readmissions and staff relationships. Focus groups will be held in facility meeting rooms and last 1 hour. Focus groups will be audio recorded and transcribed.	The mixed role compositions of the focus groups will provide opportunities for the team to document group interactions, and for the identification of group norms, differences, attitudes and priorities.50 They will provide specific information about organisational relationships and sensemaking.
Observations of care transitions work	Observations may last between 10 min (eg, patient education) and several hours (eg, medical team rounds). Investigators record their observations in field notes.28 Objective field notes will focus on interactions between people, the qualities of those interactions (eg, roles interacting, who says or does what) and how and what information is communicated. After observations are completed, investigators will fill in gaps in handwritten notes and add contextual information (eg, description of setting). Analytic notes may also be written (eg, questions for follow-up, comparing and contrasting with other data), but will be differentiated from objective data by italics or brackets. Written field notes will be taken during the observation and later typed.	Observation notes will also serve to inform the site’s care transition process checklist, as well as assessment of relationships and sensemaking.
Checklist for care transition processes	The checklist (see online additional file 2) contains items that during proposal preparation work were gleaned processes from the published papers and manuals for care transitions starting with the systematic review by Hansen et al,14 matching across studies and arriving at a comprehensive list. Care transitions on the list will be scored as present, absent or inconsistent. During the 5-day site visit, site investigators will independently fill out the checklist. At the completion of the site visit, investigators will meet to identify on a structured checklist the established care transition processes they observed and heard about during the site visit to create an agreed on version. This version will be entered in REDCap by the project coordinator.	This checklist will help us to quickly quantify how many and which care transition processes are used at each facility.
Debrief with facility leaders	Exit debriefs will consist of 40 min presentations by the project Principal Investigator and 20 min of questions and discussion with invited facility leaders. Debriefs will follow a general format1: explanation of the study and its methods2; description of care transition resources, processes and special programmes or initiatives to reduce readmissions at the site3; preliminary identified challenges to reducing readmissions and4 feedback. When possible, they will be audio recorded and detailed summary notes recorded for analysis.	Leadership debriefments provide leaders an opportunity to fill in what they might see as gaps or errors in the investigators’ understanding, to sensemake about the information presented, and to reflect on priorities and processes at their facility.
Frontline provider surveys	The survey items consist of: Work Relationship Scale developed in our previous study of learning and relationships,51 relational coordination adapted from Gittell’s healthcare work52 and an adapted version of the Safety Organising Scale as a measure of sensemaking25 (see online additional file 3). Work Relationships Scale: a 15-item scale developed to assess the perceived quality of working relationships in healthcare settings developed in a previous study by our group. We drew on the organisational behaviour literature to develop an original set of 19 items reflecting the 7 characteristics of work relationships identified among high-functioning Primary Care clinics by Lanham et al.21 The 15-item scale is associated with patient satisfaction with care in the PC environment.51 In our survey, to avoid redundancy with items from the other instruments (see below), we have reduced this to a nine items to which participants respond on a 5-point scale (from strongly disagree to strongly agree). Relational Coordination (RC) Survey: The RC survey includes questions that examine seven dimensions that were developed through inductive field research, and which have been validated in several studies. Items are rated by participants on a 5-point scale indicating the frequency to which each dimension exists in their care setting (eg, frequency: 1=never, 5=constantly). This instrument has been found to be reliable for use in airline and healthcare industries with Cronbach’s α 0.80 and 0.86, respectively.53 Adapted Safety Organization Scale: this scale measures behaviours related to sensemaking and improvising around patient safety, for example, how the team reacts to a crisis situation.25 Participants respond to eight statements, such as ‘We talk about readmissions and ways to learn from them’, using a 7-point scale (from not at all to a very great extent). This scale was developed for nursing use in inpatient setting and modifications were made to change language to be appropriate to care transitions. Participants will complete the survey on paper or through the online web application REDCap. Paper copies will be personally distributed and collected by investigators while conducting activities on site (eg, during discharge planning meetings, at interviews and focus groups). Web links to the survey will be provided through email. Completed surveys are anonymous and will not include any respondent’s personally identifiable information.	Results of this survey will be considered markers of relationships among staff participating in patient care transitions and the care transition team’s ability to make sense.

Qualitative data analysis

For each case study, qualitative analysis will overlap with data collection processes. Early findings will inform site-specific adjustments to on-site data collection protocols. Qualitative data analysis will take two forms: memoing and coding.

Memoing

The team will keep a variety of memos during data collection and analysis (table 5). Memos record reflexive comments about methods, data and theory.31 Memos will provide early opportunities for writing about and making connections within the case study data. Some memos will be written by individual researchers (eg, chart review memos), while others will be created by several researchers through discussion (eg, meeting memos, facility reflections). Memos will be periodically reviewed at team meetings to inform ongoing data collection, qualitative coding and model building. They also serve to help document team sensemaking.

Table 5

Memo types

Memo type	Description
Meeting memos	Detailed summary meeting notes will be kept during team meetings. As described by Eisenhardt,28 team meetings can be useful for overlapping data collection and analysis. These meeting notes will document, eg, how and why data collection protocols change, what researchers are learning about a specific site and how what they are learning informs theory and agent-based model building. This information will be extracted as memos.
Chart review memos	While conducting chart reviews, researchers will write memos to record and reflect on (a) care transition processes evident in the notes (eg, readmission risk assessment, discharge education, postdischarge follow-up), (b) provider communication (eg, cosigning practices, discrepancies in what providers report), (c) sensemaking (eg, providers documented concerns, how patients’ situations are described) and (d) questions or issues for team follow-up. These memos will serve to help the team document what they know so far about care transition processes at the site, identify questions for follow-up and reflect on specific cases and provider relationships and sensemaking.
Facility reflections	These one to two page documents will be written by investigators conducting the site visits during postvisit meetings. Reflections will be organised by headings derived from the agent-based model. These headings will evolve as the agent-based model develops (see below). Examples of possible headings include: institutional history and leadership, structures and routines and information flow and exchange. These analytic memos31 document and summarise what the team thinks they know about the site, what patterns they observed during data collection and what gaps might exist in their knowledge. Site reflections will inform the final site case study, data collection methods and approaches at future sites and ongoing analysis and model building (see below).

Qualitative coding

Transcripts will be analysed using NVivo software.32 We will develop a code book using deductive and inductive approaches. An initial codebook will be created based on the original model (figure 1). It will be modified as additional elements and patterns are observed through memoing, code report reading and model building.

Coding will proceed in a stepped fashion. For the first two sites, six team members (LP, JP, PN, HL, EF and the project coordinator) will code all interview and focus group transcripts. For each site, a random sample of 20% of transcripts will be independently coded by two members of the team. Pairs will check for concordance and discrepancies will be discussed by the team, and the codebook updated as needed in bimonthly coding meetings. For the final seven sites, three team members (HL, the project coordinator and a research assistant) will code the remaining transcripts. They will check for concordance on at least 10% of a random sample of transcripts for each site. Areas of discrepancy will be discussed and resolved by the full research team during weekly team meetings.

Quantitative data analysis

Quantitative data analysis will be conducted on data collected through patient chart reviews, staff surveys and observations. Knowing readmission rates can change rapidly, at the end of data collection we will also acquire from the VA data warehouse each site’s current 5-year readmission rate trend to ensure each site is correctly categorised (as improving or worsening). We will adjust categorisation as necessary. Statistical tests will be conducted in Stata IC V.14.33

Chart notes

At each site, we will determine the likelihood each note type documents the different readmission risk factors and identify which, if any, providers are usually cosigned to the note. We will evaluate findings across and within note types, and across facilities. Findings will also be compared with qualitative data (eg, interview data related to coordination practices and sensemaking related to readmission risk).

Staff surveys

The survey’s three scales will be scored as described in table 6, and the scores compared between sites. As response rates allow, some within site comparisons may also be made. Results will be triangulated with observation, interview and focus group data.

Table 6

Scoring frontline provider surveys

Survey instrument	Scoring
Work Relationship Scale (WRS)	Due to survey burden and partial overlap with other scales (see below), the original 15-item WRS was reduced to 9 items based on the original Rasch item analyses and areas of overlap with items on the other scales. Items 1, 2, 4, 5, 8, 9, 11, 14 and 15 of the original items were retained and references to clinic were changed to team.51 A new Rasch item analysis and principal components analysis will be conducted to assure that unidimensionality has been retained. Total scores will be calculated per respondent (possible range 9–45), averaged across respondents for each facility and facilities will be compared using SAS PROC Mixed.
Relational Coordination (RC) Survey	RC scores are first calculated for each individual by summing the scores of all roles (eg, care transitions staff, inpatient attending, outpatient primary care nurse, etc) for each dimension (eg, frequent communication) and then dividing by the number of responses. The overall RC score for each participant is derived by calculating the mean of the seven individual scores (range 1–5).53 RC scores at the facility level are calculated for each functional group (eg, care transitions manager, hospitalist, primary care nurse or physician) by calculating the mean of each dimension for all members of the functional group, and then a facility RC mean. The primary analyses will use the facility mean score, and secondary analyses will examine variation in RC scores among functional groups (care transitions staff, inpatient attendings, primary care teams).
Adapted Safety Organization Scale	Originally described by Vogus and Sutcliffe54 as a measure of self-reported behaviours enabling a safety culture in-hospital nursing units. Original respondents were nurses only. Questions 1, 3 and 4 were used unmodified. Questions 2, 4, 7, 8 and 9 were modified to be focused on care transitions and preventing readmissions. For example, the original question 2 was ‘we talk about mistakes and ways to learn from them’. The modified version is ‘we talk about readmissions and ways to learn from them’. The original question 5 was dropped as it dealt only with inpatient nursing shift report giving. The responses were kept the same. As for the Work Relationship Scale above, a Rasch item analysis and principal components analysis will be conducted to assure that unidimensionality has been retained. Total scores will be calculated per respondent (possible range 8–56), averaged across respondents for each facility and facilities will be compared using SAS PROC Mixed.

Observation note scoring

Within their field notes, site investigators will identify the following types of observations for structured scoring: (1) discharge planning meetings; (2) staff-to-staff interactions and (3) staff-to-patient discharge education. Notes from each observation will be entered into scoring logs and scored according to relationship and sensemaking features (table 7). The scoring systems are based on the system by Lanham and colleagues34 and situation, task, intent, concern and calibrate frameworks.35 Project staff will enter scoring into REDCap.

Table 7

Relationship and sensemaking characteristics to be scored during observations

Characteristic	Behaviours we will observe	Metric
Relationships
Trust	Saying "I don’t know" Asking for help Accepting others' clinical judgements if person is a peer or lower in hierarchy Mistrust	Interactions will be given a ‘−1,’ ‘0’ or ‘1’ based on the presence of negative behaviours, absence of behaviours or positive behaviours reflecting each relationship characteristic.
Diversity	Number/level of team members who contribute to plan
Respect	Extent to which team members listen to each other, allow each other to talk without interruption and consider each other’s suggestions
Rich/lean communication	Using verbal communication with others not in the room or with each other outside the meeting Type of communication with other staff members and with consultants
Social/task relatedness	Whether staff talk about work and non-work topics/personal lives Jokes made Laughter
Heedful inter-relating	Acknowledging the potential/actual impact of their behaviours on how others get their jobs done or on patient care or disposition planning
Mindfulness	Responding to each other’s ideas for the evolving plan Helping each other with tasks Suggesting new ideas or discussing how the team might do things differently
Sensemaking
Situation	Assesses patient’s situation	Teams will be given a ‘0’ or ‘1’ based on the use or non-use of each sensemaking element.
Task	Develops a plan about what needs to get done (objectives) based on assessment of patient
Intent	Statement of rationale for the plan
Concern	Discusses concerns/things that could go wrong/things where plan might fall short with patient. Develops a contingency plan
Calibrate	Asks for feedback from each other about the plan based on concerns
Social vs solitary	Shared decision-making between staff, patient and/or family. May be between two staff members. Must come to a shared understanding
Degree of identity definition	Performs tasks outside of hierarchical role
Backward-noticing	Discussion of prior patients with similar presentation or issues, or prior situation of the current patient

Two investigators experienced with applying these frameworks to observations in medical settings (LL and HL) will train the team on how to recognise behaviours that match these characteristics. Consistency in scoring will be established through use of the codebook and during multiple rounds of team scoring. For the first two sites, during weekly meetings following data collection, a sample of roughly 5% of the observations will be independently scored by each team member. Scoring will be compared and discrepancies discussed until the group has reached consensus. Clarifying discussions about scoring will be documented in meeting notes and fed back to improve the scoring guide. Visual inspection of the distribution of all variables will be performed. Where appropriate, power transformations will be applied to variables outside of assumptions of parametric statistics. Group differences will be determined using ordinary or generalised least squares regression with the relevant covariates.

Objective 2. Creating, verifying and validating an ABM of sensemaking regarding transitions of care and prevention of readmissions

Complex, non-linear systems are difficult to study with traditional analytic methods because of multiple interactions among variables, feedback loops, path dependency and contingencies in any dynamic process; there is often no set of equations that can be solved to predict characteristics of the system.36 A more effective way to examine non-linear behaviour in complex systems is to simulate it by building a model and then running the simulation multiple times to explore the space of possible system trajectories.36 In our study of sensemaking and readmissions, the interdependencies among the patients, healthcare providers, resources (VHA and non-VHA) and leadership support are clearly non-linear. Individuals who make sense of the ways in which readmissions occur illustrate this by mentioning different aspects they consider to be critical: patient context, patient understanding and motivation, resource availability, effective communication between healthcare providers, stage of disease, failures in a system for which they (patient or provider) have little control. These aspects interact in variable ways in the context of different patients. Vest et al identified the plethora of variables that contribute to readmissions before even addressing the interdependencies.37 Additionally, the literature demonstrates that classical prediction models of readmissions perform poorly.12 We suggest that these explanatory gaps in the literature are due at least in part to a mismatch of analytic strategy to type of system being studied. We see readmission as an emergent outcome of non-linear interactions among these many aspects of clinical and organisational processes. Through modelling and simulation, we will be better able to understand and evaluate factors contributing to readmissions. While any single case may be difficult to predict, modelling will allow us to identify leverage points in the system that the data demonstrate are particularly sensitive to sensemaking effectiveness. These leverage points could then be considered potential targets for interventions. Through modelling and the subsequent ability to run it numerous times (simulation), we will be able to extend the case study sample to make it more generalisable to better understand how readmissions occur across the care transition interventions, patient circumstances and facility environments. Through modelling and simulations we are able to create a laboratory that will allow us to understand better how readmissions occur, helping us to identify gaps in our knowledge as well.

ABM is a version of non-linear dynamic modelling, a computer implementation of complexity concepts, in which autonomous agents interact in an environment to produce emergent—sometimes surprising—system properties over time.38–40 Since the pioneering work by Epstein and Axtell in the late 1990s,41 it has been applied to research on human groups under the rubric of ‘artificial societies’.36 ABM is an ideal approach to our research questions for several reasons: first, as noted earlier, our data regarding healthcare provider interactions are non-linear, making it potentially more difficult to represent patterns and interdependencies using more traditional approaches. ABMs are grounded in non-linear mathematics, assuming interactions and contingencies in a manner that more accurately reflects clinical systems. Second, ABMs allow us to create a broader space of outcomes from rich observations that may be low in number but high in information, accounting for the facilities and teams within facilities that we sample, and other types of findings that result from experimenting with parameter changes. Formalising the interactions leads to a generalisation of the processes we observed. Thus, ABMs enable us to leverage small samples to create broader understandings. Third, we can model interactions across levels and over time to explore emergent outcomes. ABMs are laboratories for structure-agency interactions that allow us to understand these multiple levels.

Proposed modelling work

Conceptual work

While data are being collected, our research team will meet regularly to identify the parameters, agent characteristics and interaction patterns. Our starting point will be the conceptual model of care transitions shown in figure 1. As we develop the ABM, we will iteratively build on our conceptual model using the qualitative data being collected. We will begin developing the ABM after our first few site visits, and refine the model with each subsequent visit. Constructing the model in this way will complement our qualitative data collection and help us identify areas where more intensive inquiry might be necessary. Initial tasks for building the model will include identification of the following:

Types of agents to be included

In ABM agents can and, in our case, will have correspondence to real-world actors, both individuals and organisational units. We will start with the general categories of patients, inpatient providers, outpatient providers and care transitions personnel. We will then refine the specific individuals contained in these categories, and add any additional categories or types of individuals as we collect and analyse our qualitative data.

Interactions and interdependencies among agents

We will create rules of interaction between the agents in the model based on our site visit data, starting with the initial site visits and refining these interactions with subsequent site visit data. Interactions will focus on the sensemaking activities and categories we observe in the site visits. These sensemaking attributes are detailed in the’Observations of care transitions work' and ’Qualitative data analysis' sections.

Boundaries and characteristics of the environment

Our model will be built to simulate a single organisational entity. We will create a model to allow ourselves the ability to adjust these characteristics and assess their impact through our simulations. We intend to simulate critical facility characteristics and will use the first year to consider the types of qualitative characteristics we will obtain during the site visits as well as the quantitative data already available for VHA facilities such as culture (annual employee survey), learning and improvement culture (voice of VHA survey), number of care transition processes used routinely (from our prior UM survey and verification for study sites), demographics of veterans served and facility admission rates. We will also consider known parameters used in traditional readmission prediction models, although most of these parameters focus on the patient such as comorbidities, prior healthcare use, functional status, socioeconomic status.12 37 Organisational characteristics relate back to the technical processes of care and system resources noted on our conceptual model.

Levels of model

One of the rationales in studying transitions of care as an exemplar is the multiple individuals and teams that interact with the patient and the system to make the care transitions successful. A benefit of ABM is that it allows us to consider levels of interactions, and the system-level outcomes that emerge from these levels of interactions. In building the model, we will need to address how different parts interact with the next to produce the product of interest—successful or unsuccessful care transitions. Care transition teams and Veterans interact with inpatient teams as well as outpatient teams, resource providers (such as prosthetics and pharmacy), home care providers, institutional providers and patient caregivers. Additionally, leadership determines extent of resources available at many of these levels. We will define the levels and how they will feed into each other. Again, we will use our conceptual model of care transitions as the starting point. Processes of care and the organisational characteristics will form this level. The formal interactions or organisational structure will also be reflected here. The agents will interact in this level, producing emergent outcomes of sensemaking that are grounded in their interactions and inter-relating. These sensemaking patterns will form the second level of the model. From them, care transition outcomes will emerge, forming the model outputs. In our model, the two outcomes will be a successful care transition or a readmission.

Feedback loops can be created within the levels of the model. For example, as either successful care transitions or readmissions occur, these outcomes can feed back into how the agents’ sensemaking processes. We will specifically collect data on these types of feedback loops during our site visits. (See table 4 and additional file 1.) These feedback effects will be modelled using standard best practices from the system dynamics modelling methodology, which concentrates on how to model systems with non-linear feedback loops.42–44

Modelling software:

We will use NetLogo software to create our model. NetLogo is a freely available software that has been under development for two decades and is widely used for ABM.45 It is now in V.5 and has become a sophisticated language for modelling intelligent autonomous agents interacting in ‘live’ environments. With the most recent versions, NetLogo extensions have been incorporated that enable more sophisticated agents and with hybrid capabilities enabling combined agent-based and discrete-event simulation. These capabilities will allow us to create a robust model that best represents the relevant processes of care and agent interactions.

Model verification and refinement

As we develop the model, we will make our understanding of the interdependencies between different levels more explicit. Because we will begin to conceptualise and create the model in parallel with data collection, we will be able to use ongoing site visits to refine aspects of our model.

Additionally, we will perform verification to ensure that the associations and interdependencies between levels of the model are expressed in the way we intend. Verification ‘concerns whether the programme is working as the researcher expects it to’.36 Our model will act as a thought-experiment laboratory that forces us to clarify and formalise the interactions in which we are interested. The verification will support this clarification.

Model simulation and sensitivity testing

We will use simulation to deepen our understanding of the ways that provider sensemaking influences care transition outcomes. We will be able to vary the following parameters: organisational factors, including patient population characteristics and other facility-level data; care transition practices; sensemaking practices. We will assess the impact of parameter variation on our outcome of interest—readmissions and successful care transitions. During this time, simulations will be run for multiple ‘facilities’ to expand the generalisability of our qualitative sample, using different combinations of individual and facility characteristics to understand how sensemaking emerges, and how sensemaking then impacts care transition outcomes.

Model verification and boundary testing

During this period, we will present our model results to our local site PIs from 10 sites as well as our systems re-engineering organisational partners for input as to the face validity of the findings of the simulations. These presentations will follow a formal, focus group process to ensure that we capture all concerns and feedback regarding the model. We will use this feedback to further refine the model.

Study status

Data collection at the first case study site began in July 2015 and continued through December 2017. Qualitative and quantitative data analysis, and ABM work began during this period and were ongoing at the time of writing.

Ethics and dissemination

Participation in this study is voluntary and participants are not compensated for their participation. Written consent and Health Insurance Portability and Accountability Act forms are obtained for patients participating in interviews. As permitted by our Institutional Review Board, VA staff participating in research activities (eg, interviews, surveys, observations) are given an information form about the study, assured confidentiality and asked to give verbal consent to participation.

Findings from our work will be disseminated through manuscripts in peer-reviewed journals, at professional conferences and in short reports distributed to stakeholders and study participants. Our data will not be made available in repositories.