A mixed-methods analysis of telehealth implementation in nursing homes amidst the COVID-19 pandemic.

BACKGROUND: The COVID-19 pandemic has forced nursing homes to adapt new models of care in response to the evolving crisis including rapid implementation of telehealth services. The purpose of our study was to investigate implementation of telehealth in nursing homes amidst the COVID-19 pandemic using a human factors model.
METHODS: Using a mixed methods design, we conducted a secondary analysis of data from a national survey of nursing home administrative leaders (n = 204). Using six survey questions, we calculated a total telehealth score (range 0-42). Descriptive statistics and paired sample t-test were used to explore the change in telehealth in two consecutive years (2019-2021). Next, we conducted semi-structured interviews with (n = 21) administrators and clinicians to assess differences in implementation according to extent of telehealth use.
RESULTS: The mean telehealth score in year 1 was 12.11 (SD = 9.85) and year 2 was 19.25 (SD = 11.25). There was a significant difference in telehealth scores from year 1 to year 2 (t = 6.83, p < 0.000). While 64% of nursing homes reported higher telehealth scores in year 2 compared to year 1, over 32% reported a decline. Qualitative analysis revealed facilitators of telehealth including training, use of integrated equipment, having staff present for the visit, and using telehealth for different types of visits. Barriers included using smart phones to conduct the visit, billing, interoperability and staffing.
CONCLUSION: Training, adaptation of work processes to support communication, and restructuring teams and tasks are the result of interactions between system components that could improve usability and sustainability of telehealth in nursing homes.

Key points

Among US nursing homes responding to two consecutive years of the health information technology maturity survey (2019–2021), telehealth use increased significantly.

Respondents suggest training, adaptation of work processes to support communication, and restructuring teams and tasks are factors that could improve usability and sustainability of telehealth in nursing homes.

Why does this paper matter?

As clinicians, policy makers and other stakeholders consider sustained use of telehealth for vulnerable nursing home residents, they should consider facilitators and barriers to telehealth implementation that may impact healthcare quality and patient safety.

INTRODUCTION

The ongoing COVID‐19 pandemic has affected over 276 million people worldwide and led to over 5.3 million deaths, resulting in societal and economic turmoil of unparalleled magnitude. Staff and residents at long‐term care facilities, including nursing homes (NHs) were particularly hard hit during the first year of the pandemic, accounting for 31% of all COVID‐19 deaths in the United States. Ensuring the safety of NH residents while maintaining access to care led to the rapid adoption of telehealth (TH) solutions. TH is defined as the delivery and facilitation of health and health‐related services including medical care, provider and patient education, health information services, and self‐care via telecommunications and digital communication technologies. To help maintain access to care amid stay‐at‐home orders intended to reduce COVID‐19 related exposure, the Centers for Medicare & Medicaid Services (CMS) used emergency waiver authorities enacted by Congress to implement policies expanding access to TH services during the pandemic. These included waiving several statutory limitations such as geographic restrictions, allowing beneficiaries to receive TH in any location including NHs. Additionally, the Office for Civil Rights relaxed enforcement of Health Insurance Portability and Accountability Act (HIPAA) of 1996 privacy requirements for videoconferencing.

While the pandemic created urgency for individuals and healthcare systems to shift to TH, it has been used across many sectors of the health care system, including NHs, for decades. , In many ways, NHs represent ideal settings for the use of TH services. NH clinicians may be physically present at the NH on an intermittent basis, creating challenges contributing to misdiagnoses, delayed diagnoses, overuse of emergency department visits, and avoidable hospitalizations. , Research has shown many potential benefits associated with TH use in NHs, such as expanding access to care, addressing gaps in coverage, reducing the burden on staff, improving resident outcomes, and achieving cost savings. Despite these potential benefits, poor software engineering, security breaches, and exacerbation of existing disparities in resident care have been found to occur. Optimization and sustainability of TH requires solutions that are easy to use and maintain.

Human factors is the practice of fitting the work system to the human, not the other way around. It focuses on designing various factors of the system according to how humans think and behave rather than forcing the human to adapt their behavior to the system. Analysis of rapid technology implementation usually demonstrates different underlying systems' shortcomings including human factor issues impacting usability and sustainability. Therefore, the application of human factors engineering and systems engineering concepts has been proposed to improve healthcare quality and patient safety. One systematic review found only 15 articles reporting TH system evaluation with user input, and of those 15, only four actually measured usability. The review concluded that stakeholders, such as providers, patients, nurses, and support staff, should be central to the TH design process.

The purpose of our study was to investigate implementation of TH in NHs amidst the COVID‐19 pandemic from a human factor's perspective, specifically, using the System Engineering Initiative for Patient Safety (SEIPS) model. The SEIPS model, anchored in human factors and systems engineering, provides a framework for comprehensive and proactive assessment of TH implementation and helps to determine key interactions between system components: person, technologies, environment, tasks and organization. Person is defined as the education, knowledge and motivation as it relates to patients and clinicians. Technologies are devices and information systems individuals use to deliver care. Environment refers to environmental factors (e.g., workstation design, layout, noise) occurring where care is delivered. Tasks are the specific actions taking place within a larger work process. Lastly, the organization component emphasizes teamwork, collaboration, communication and organizational culture.

The SEIPS model has been used in various settings including intensive care units, outpatient surgery centers and in cardiac operating rooms to guide the assessment of systems, processes and outcomes. , To our knowledge, our research team is the first to apply the SEIPS model to help understand TH implementation in NHs. In this study, we used a mixed methods approach to address the following research questions: (1) Did NHs experience significant change in TH utilization in two consecutive years? (2) What human factors issues did clinical staff experience during TH implementation in NHs with diverse stages of adoption?

METHODS

Study design

A mixed sequential design was used beginning with a quantitative phase measuring the extent to which TH use changed in NHs pre‐ and post‐modified TH regulations (March 6, 2020) during the COVID‐19 pandemic. This was followed by a qualitative phase consisting of semi‐structured interviews with NH leaders and clinicians who had completed surveys about TH adoption. The rationale for this mixed‐methods approach was to use qualitative methods to help explain quantitative results and to develop a rich and nuanced understanding of TH use in NHs not possible through quantitative methods alone. The following section describes methods for both the quantitative and qualitative phases of the study.

Quantitative

To explore TH adoption, we used a survey designed to measure NH health information technology (HIT) maturity, defined as the extent to which facilities possess and use diverse technological devices and software that are integrated across resident care, clinical support and administrative activities. The survey has been tested previously and determined to have good reliability and validity measures. Cronbach alpha for the total HIT maturity survey was 0.86. In addition to psychometric evaluation, the survey has been evaluated for content validity, specificity and sensitivity by a panel of NH administrative leaders.

NHs were randomly selected from each state using the Care Compare dataset (formerly known as Nursing Home Compare). Inclusion criteria were all NHs in the data set located within the United States. A total of 13,958 NHs met inclusion criteria. The target goal of the sample was 10% of NHs which were randomly selected from each state using homes listed in Care Compare. In our first national study, we achieved a 45% response rate for contacted homes that initially agreed to participate in the study. Therefore, in order to reach our recruitment goal, we randomly recruited facilities from each state until we had reached 20% of the facilities in each state who agreed to participate in the study. NHs were not stratified relative to size, location, or ownership prior to randomization to allow every facility equal opportunity to participate in each state. NH administrators were asked to select site respondents to complete the survey that had knowledge and/or oversight of IT systems in their facility. The respondent was sent an electronic (via RedCap) or paper survey after agreeing to participate. The team conducted at least three follow‐up calls per facility at 2‐week intervals to increase response rates. Researchers used double‐entry procedures entering each paper survey received into separate excel files and then cross‐checking files for accuracy before uploading into RedCap. The recruitment period used for year 1 (Y1) was February 2019–March 5, 2020 and year 2 (Y2) was March 6, 2020–February 2021. We used March 6, 2020 as the split between Y1 and Y2 as this was the date CMS enacted emergency waiver authority to modify telehealth regulations amidst the COVID‐19 pandemic.

To answer our first research question, we used six questions from the HIT maturity survey related to TH (see Table S1). Respondents were asked to rate these items according to their extent of use on a zero‐to‐seven scale, with zero indicating “not at all” and seven “very much.” The same six survey questions were used in prior work to explore TH uptake in NHs at the start of the COVID‐19 pandemic. For the current study, we calculated a cumulative TH score using the six survey questions for each home with a minimum score of zero and a maximum of 42. Psychometric properties such as reliability (Cronbach's alpha) enhance our assumptions about the use of the instrument, however, clinical relevance of the minimal changes in score should also be assessed. We used a measure to estimate the minimal change in score considered relevant by patients and providers called the minimal clinically important difference (MCID). We calculated the MCID by multiplying the pooled standard deviation by 0.2 resulting in 2.2. Based on this calculation, we conclude difference in TH score of ±2.2 should be considered clinically relevant in the analysis of this sample. Descriptive statistics were used to characterize the sample using frequencies, means and standard deviations. Further, we compared the cumulative TH score of NHs who completed the HIT maturity survey in two consecutive years using paired sample t‐tests.

Qualitative

To answer our second research question, we used a maximum variation approach to select participants who had a net positive and net negative change from Y1 to Y2 in their TH score. Our rationale for this approach was to elicit factors contributing to successful implementation (facilitators) as well as factors contributing to unsuccessful implementation (barriers). Our recruitment strategy began by ranking NHs according to the difference in their Y1–Y2 TH score. Next, we purposively recruited NHs from two groups: the positive group included any NH with +1 or greater change from Y1 to Y2 and the negative group included any NH with −1 or greater change from Y1 to Y2. Beginning with the NHs that had the largest net change from Y1‐Y2, we contacted the NH administrator from those facilities and asked for the person responsible for TH implementation at the NH. We invited that person to participate in an interview and then used a snowball approach to identify other “end users” of TH. Administrators identified these end users as clinicians (physicians and/or nurses) therefore we extended an invitation to those individuals to participate in the study.

An interview guide (see Table S2) was used to ensure consistency among participants and all interviews were conducted using Zoom video conferencing. Each interview was transcribed verbatim by members of the research team with prior transcription experience. To ensure accuracy, a second reviewer verified each transcript with the audio recording. Since we were interested in examining these data within the context of the SEIPS model, we used directed content analysis. To reduce bias and increase trustworthiness, the coding process began by reading each transcript and highlighted all text that on first impression appeared to fit within one of the components (themes) of the model. Exact words from the text were used to create codes that appeared to capture key thoughts or concepts. Next, codes were sorted into categories which were eventually grouped according to subtheme resulting in the hierarchical structure of themes, subthemes, and categories. We began data analysis after the first six interviews to estimate saturation as described by Guest et al. The iterative process of data collection and analysis continued until the new information threshold reached zero, that is, no new information was collected from subsequent interviews. After reaching the information threshold of zero we recruited three additional NHs to ensure threshold levels were achieved. Dedoose qualitative software was used to help organize and visualize data.

Processes to ensure trustworthiness were maintained by ensuring credibility, dependability, confirmability and transferability. Credibility was addressed through member checking, that is, deliberate probing during data collection to ensure the participants' meanings were understood. An audit trail was maintained for dependability and an interview guide was used to maintain consistency. Confirmability was strengthened by having two members of the research team independently code excerpts to ensure intercoder reliability. Transferability was addressed by purposely recruiting a broad range of participants with respect to TH score and NH characteristics. The study was approved by the University of Missouri Institutional Review Board project number 2009109.

RESULTS

Quantitative results

In Y1, we contacted 4699 NHs and 1303 agreed to participate in the study. We received a total of 491 surveys in Y1 (final Y1 response rate = 491/4699 = 10%). For Y2, we started with the 491 NHs that completed the Y1 survey. We received completed surveys from 337 NHs, thus we had follow up responses from 337/491 (69%). Since we were interested in comparing differences in TH use before and after regulations were relaxed by CMS, we only included surveys in our final sample that were returned within the specified time period (Y1: February 2019–March 5, 2020; Y2: March 6, 2020–February 2021). A total of 204 NHs completed both a Y1 and Y2 survey during this timeframe. Characteristics of the final sample (n = 204) are displayed in Table 1. The distributions of TH scores for Y1, Y2 and difference between Y1 and Y2 are shown in Figure 1.Over the two year study, 129 NHs reported a positive net change in TH scores. Conversely, 66 NHs reported a net negative TH score indicating that they used TH less in Y2 compared to Y1. Nine NHs had no change in TH score from Y1 to Y2. To test the hypothesis that the Y1 TH score and Y2 TH scores were different, a paired sample t‐test was performed. Prior to conducting the analysis, the assumption of normality was examined. The assumption was considered satisfied, as the skew and kurtosis levels were estimated at −0.17 and 0.13, respectively. The mean TH score in Y1 was 12.11 (SD = 9.85) and Y2 was 19.25 (SD = 11.25). The null hypothesis of equal TH score means was rejected t = 6.83, p < 0.000. Cohen's d was estimated at 0.5 (95% CI 0.33, 0.62) which is a moderate effect size. We interpret the change in TH score from Y1 to Y2 to be clinically significant as the difference in means (7.14) exceeds the minimal clinically important difference, calculated to be 2.2.

TABLE 1

Survey respondent characteristics

	Sample (N = 204)	NHs with positive TH change (n = 129)	NHs with negative TH change (n = 66)
Location
Metro (population > 50,000)	128 (62.7%)	74 (57%)	48 (73%)
Micro (population 10,000‐49,999)	31 (15.2%)	22 (17%)	7 (11%)
Small town (2500‐9999)	22 (10.8%)	15 (12%)	6 (9%)
Rural (< 2500)	23 (11.3%)	18 (14%)	5 (7%)
Bed size
In >120 beds	36 (18%)	24 (19%)	10 (15%)
60–120 beds	120 (59%)	70 (54%)	46 (70%)
<60 beds	48 (23%)	35 (27%)	10 (15%)
Ownership
For‐profit corporation	127 (62%)	77 (60%)	44 (67%)
Non‐profit corporation	77 (38%)	52 (40%)	22 (33%)
Mean TH score
Year 1	12.11	8.12	19.88
Year 2	19.25	24.23	10.45

Note: (n = 9) NHs had a net neutral change from year 1 to year 2 and were not included in either the positive or negative group.

FIGURE 1

Distribution of change in TH score from year 1 to year 2

Qualitative results

To answer our second research question, we conducted interviews with (n = 21) NH administrators and clinicians representing 16 unique NHs. Participant demographics are presented in Table 2. Since we were interested in learning about factors that contributed to successful and unsuccessful TH implementation, we created two groups: the positive TH score group and the negative TH score group. Participants in the positive TH score group had a minimum of 8 and maximum of 25‐point increase (range of +8 to +25) in their TH score from Y1 to Y2. Participants in the negative TH score group had a minimum of 2 and maximum of 12‐point decrease (range of −2 to −12) in their TH score from Y1 to Y2.

TABLE 2

Qualitative sample characteristics

Participant title	n
Administrator	15
Clinician (nurse or physician)	6
Number of years in current position:
Range(median)	1.5–24 (3.5)
NH facility characteristics
Bed size
Small (<60)	4
Medium (60–120)	10
Large (>120)	7
Location
Metro	12
Rural	9
TH score group
Positive	10
Negative	11

Over the course of the analysis, 96 open codes were developed. Codes were combined into 14 categories and organized according to five human factors components derived from the SEIPS model: (1) person, (2) technology, (3) physical environment, (4) tasks, and (5) organization. We found similarities and differences in TH implementation by participants from both groups (positive and negative). We categorized (see Table 3) these differences in TH implementation as facilitators or barriers. A summary of themes, categories, illustrative quotes and frequencies (by facility) can be found in Table S3.

TABLE 3

Perceived facilitators and barriers to telehealth implementation in nursing homes

Facilitator/barrier by SEIPS component	Code	Illustrative quote
Person
Facilitator	Training for TH	“The company [nursing home] itself is very progressive and it's a small company. They got it set up in most of our own facilities. They okay'd it and they provided the training for it.” (admin, positive TH score)
Technology
Facilitator	Integrated Equipment	“We put telemedicine units in the facility, allowing us to see patients, digital stethoscope, listening to heart and lungs, otoscope. It made an incredible difference.” (clinician, positive TH score)
Barrier	Smart phones	“It's challenging to use an iPhone, to keep it in the right place, and to have them to be able to see the physician, and the physician to be able to see everything he needs to see because I know from when my mother was in the nursing home and we did some tele‐visits, I spent more time looking at the ceiling than I did my mom.” (clinician, negative TH score)
Environment
Facilitator	Having NH staff present for TH visits	“Another one the benefits is we [nurses] get to see and be a part of the visit, whereas when we sent them out, we hope that they read the paperwork that they sent, and then we hope that we get something coming back, so I think it gets more a connection and that's a good thing.” (clinician, positive TH score)
Tasks
Facilitator	Using TH for different types of visits	“Last week we had one resident who was very acutely sick and so, our director of nursing had our medical director's partner on FaceTime in the room just kind of assessing from afar.” (admin, positive TH score)
Organization
Barrier	Billing for TH	“So probably the biggest issue that we have is getting paid for it.” (clinician, positive TH score)
Barrier	Interoperability	“Our lab has a portal that we use, but providers do not really get into it. So that is a problem, you are literally typing in whatever their app is, blood pressure was this and their vitals. That's difficult because providers use so many different things in order to do their visits.” (admin, negative TH score)

Abbreviation: SEIPS, System Engineering Initiative for Patient Safety.

Person

Person was defined as the education, knowledge and motivation of those using TH. This theme includes two categories: training for TH and motivation. Only NHs from the positive TH score group reported having any formal training to use TH. Sources of motivation for using TH were diverse among participants from both groups and included improving resident safety, transportation associated cost savings, improved communication, improved quality of care, and saving time for the provider. Among these motivating factors, improved communication and improved quality of care were perceived more often by participants from NHs who had increased use of TH from Y1 to Y2.

Technology

Technology was defined as devices and information systems used to conduct TH visits. This theme included the categories hardware and software. Hardware refers to the device used by NHs to conduct TH visits. All NHs in our sample reported using a tablet (iPad) for TH visits. Laptop computers were also used by NHs from both groups. Integrated equipment which included tools like stethoscopes and otoscopes were only used by NHs from the positive TH score group. Further, only NHs from the negative TH score group reported using a smart phone for TH visits. Most NHs in our sample (n = 12) used videoconferencing software such as Zoom, Skype, and FaceTime to conduct TH visits. However, six NHs (three from the positive group and three from the negative group) reported using software developed specifically for TH. Patient portals were used by two NHs in the positive TH score group to conduct TH visits.

Physical environment

Physical environment refers to characteristics of noise, workstation layout and available space associated with TH visits. This theme includes three categories: setting up TH visits, who is present during the visit, and space for conducting TH visits. Most NHs (n = 12) reported they used nursing staff from the NH to set up TH visits with residents. Three NHs reported a staff member from the provider's office (medical assistant) set up the visit in the NH. One NH from the negative TH score group used NH administrative staff to set up TH visits. Participants reported differences in who was physically present during the TH visit. In most cases a member of the NH staff was present with the resident for the TH visit. Only NHs from the negative TH score group reported they typically conducted visits with only the resident present. Two NHs (one from each group) reported having a medical assistant present with the resident for TH visits. The final category under the theme of physical environment is the space where TH visits took place in the NH. Most NHs in our sample reported having a mix of private and semi‐private resident rooms where they conducted TH visits. However, five NHs reported using another room in the NH such as a consultation room or activities room to conduct TH visits.

Tasks

Tasks were defined as the content, participation and demands of TH visits. This theme included one category: the type of visit conducted using TH. All NHs reported using TH for consultation/specialist visits and most used TH for routine or follow‐up visits. TH was used in situations where the resident had a change in condition more often among NHs from the positive TH group compared to the negative TH group.

Organization

The theme of organization describes teamwork, coordination, collaboration, and organizational culture impacting TH use. This theme includes five categories: pre‐COVID use of TH, billing for TH, documentation of TH visits, interoperability barriers, and staffing. Only four NHs (three positive and one negative) reported using TH in any capacity pre‐COVID. Further, we asked participants about billing for the technical component of TH visits, as this was allowable under the CMS waiver. Only NHs from the positive TH score group, reported billing for TH visits. In most cases documentation of the TH visit was done by the nursing staff using the NHs electronic medical record. Participants from five NHs indicated they were able to document the TH visit directly in the NHs electronic medical record. Interoperability was perceived as a barrier by many participants in our study. Participants reported difficulty with disparate systems requiring multiple logins to access information. For example, providers reported difficulty remotely accessing systems containing lab results, vital signs, and progress notes. Finally, the category of staffing refers to comments made by participants about how TH has impacted staffing in NHs. All NHs in our sample reported that increased use of TH had increased what NH staff were required to do (e.g. setting up, facilitating, and documenting TH visits), however, none of the NHs reported adding staff to accommodate for the increased use of TH.

DISCUSSION

In this study, we explored implementation of TH in NHs amidst the COVID‐19 pandemic using a mixed methods approach. Quantitative analysis revealed differences in TH use over a two‐year timeframe. While some NHs (n = 129) increased the extent to which they used TH, others (n = 66) reported a decrease from Y1 to Y2. This finding suggests that despite regulatory changes intended to simplify and expedite TH implementation, including financial incentives and relaxed regulatory oversight, use declined in some NHs. Quantitative analysis alone does not provide complete information about the decline of some NHs TH score; however, our qualitative results provide some insight. Situating our findings within the SEIPS model offers some explanation as to how some NHs were able to fit the work system to the human (i.e. facilitators) and barriers encountered by others. Training, adaptation of work processes to support communication, and restructuring teams and tasks are the result of interactions between system components that should be considered for improving usability and sustainability of TH in NHs.

In our study, only NHs who experienced a positive change in TH score reported any training, thus we consider this to be a facilitator of use. Training is impacted by multiple system components in the SEIPS model. For example, training impacts the user of TH (person) based on the tasks they perform in the physical environment. Dubose‐Morris and colleagues reported similar findings in a study evaluating education and training implemented in response to the COVID‐19 pandemic. Among their sample of medical facilities (hospitals and physician practices), 30% reported no formalized training and 95% of those who reported no training pre‐pandemic reported implementing TH training as part of the pandemic response. The importance of TH training is supported by a recent qualitative study where behavioral health students expressed desire for additional training including opportunities to practice TH visits prior to meeting with patients. While rapid TH implementation was necessary in NHs during the initial peak of the COVID‐19 pandemic, improved usability and sustainability rely on further research including best practices for staff training and identifying clinical competencies for TH use.

Adaptation of work processes to enhance communication could improve effectiveness of TH in NHs. According to the SEIPS model, work processes are shaped by the interaction of system components and describe how the work is done, in this case, TH visits. Participants in our study reported difficulty conducting TH visits due to poor data integration and inconsistent documentation of TH visits. While NHs have made progress in data sharing and integration, interoperability continues to be a barrier. , , In the current study, we expected to find differences in the type of device used for TH visits, however, our finding that only NHs in the positive TH score group were using integrated equipment was noteworthy. Further research is needed to determine if the type of device used (i.e., integrated equipment vs. tablets and smartphones) is associated with enhanced usability and ultimately improved resident outcomes. In the meantime, NHs should consider available TH workflow optimization strategies including the use of checklists and toolkits such as the CMS NH TH toolkit which contains fact sheets and checklists on topics such as setting up TH, selecting a vendor, and technical assistance.

Increased TH uptake in NHs has required restructuring the interaction between tasks and teams. The majority of NHs in our sample were not using TH prior to the COVID‐19 pandemic. After CMS enacted emergency waiver authority, all NHs in our sample were using TH, albeit for different tasks. For example, all 16 NHs in our qualitative sample used TH for consultation/specialty care and most (81%) used it for primary care. A recent report from the U.S. Department of Health and Human Services (DHHS) found the number of Medicare visits conducted via TH increased 63‐fold from approximately 840,000 in 2019 to 52.7 million in 2020. Similar to our findings, the report showed from 2019 to 2020, specialty care had the largest increased in TH (38‐fold increase) followed by behavioral health (32‐fold increase) and primary care (24‐fold increase) respectively. As NH leaders and policy makers evaluate TH use in NHs beyond the pandemic, they should consider how different types of use impact resident and organizational outcomes.

None of the NHs in our sample reported adding staff to accommodate for the increased use of TH, however, all participants noted changes in responsibility. According to our participants, nursing staff were tasked with additional responsibilities (setting up, facilitating, and documenting visits) as TH use increased. A recent analysis using CMS payroll data found no significant changes in NH staffing levels in the first 9 months of 2020 compared with the same time period in 2019. Adding tasks related to TH and other measures, such as infection control (e.g., providing care typically provided by informal caregivers, dedicated staff to care for residents with COVID‐19) could lead to stress and burnout. Minimizing staff burden associated with TH is an imperative and should be examined in future work.

This study has several limitations. First, because we used secondary data for the quantitative portion of the study, analyses were limited to TH data available in the NH HIT maturity survey. Second, although we used methods to enhance generalizability of qualitative results, it should be considered a limitation as the qualitative participants were based on a convenience sample. Further, we did not include residents in this evaluation because our focus was on the work system. Resident input is important and should be considered in future studies to understand how satisfied they are with the interaction and treatment resulting from the TH encounter. Finally, causality should not be implied as this was an exploratory mixed methods study.

AUTHOR CONTRIBUTIONS

All authors participated in study concept and design. KP and GLA participated in acquisition of subjects and or/or data used in the analyses. All authors participated in analysis, interpretation of data and manuscript preparation.

CONFLICT OF INTEREST

The authors have no personal conflicts to disclose.

SPONSOR'S ROLE

AHRQ funded the research team to administer the IT maturity survey.

FINANCIAL DISCLOSURES

This study was supported by grant R01HS02249 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ (PI: Alexander).

Table S1 Questions from the Nursing Home Information Technology Maturity Survey used to calculate the total telehealth score.

Table S2. Qualitative interview guide. Introductory prompt: tell me about how you/your nursing home was using telehealth before the COVID‐19 pandemic and now you are using it today.

Table S3. Qualitative results.

Click here for additional data file.