Human Connection and Technology Connectivity: A Systematic Review of Available Telehealth Survey Instruments.

CONTEXT: The current upsurge in telehealth use in palliative and hospice care warrants consideration of patient, family caregiver, and interdisciplinary palliative perspectives on telehealth modality and communication experiences. Currently, telehealth experiences and encounters are being described but not yet extensively evaluated by palliative care teams.
OBJECTIVES: To locate survey instruments available to assess telehealth interactions, to determine the content and constructs covered by the available instruments, and to describe the patient populations previously surveyed by the existing instruments.
METHODS: This study and its reporting followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines with the protocol registered in The International Prospective Register of Systematic Reviews. Three databases were searched with over 3100 articles analyzed for use of a telehealth survey instrument.
RESULTS: Twelve telehealth communication assessment instruments were identified with a mean length of 20 questions, primarily Likert-scale responses with one inclusive of free text and one qualitative inquiry survey. Three inquired only into modality, four queried communication, and five studied both modality and communication experience. Existing telehealth survey instruments are unidirectional in exploring patient or family experience, with two inclusive of provider perspectives. Participant demographics are notably underreported in telehealth experience studies with a frank lack of diversity in ethnic/racial, geographic, age, educational, and income representativeness in current telehealth survey instrument respondents.
CONCLUSION: Palliative care teams may consider familiarity with telehealth survey instrument as an essential component to progress from description of telehealth use to evaluation of telehealth encounters. Current survey instrument outcome reports do not represent inclusivity or diversity, although telehealth is now being clinically applied across settings.

Key Message

This systematic review searched >3100 studies for telehealth communication assessment tools, leading to the identification of 12 available telehealth survey instruments. This work describes their format, content, constructs, psychometric properties, and application. These instruments have been largely studied in white, formally educated, metropolis-centered populations as opposed to populations more reflective of diverse communities.

Introduction

The COVID-19 pandemic has resulted in a rapid surge of telehealth use in palliative and hospice practices.1, 2, 3 Before the coronavirus crisis, telehealth uptake in palliative care and hospice had been slow and understudied.4, 5, 6 Telehealth has historically been deployed to decrease distance between patients and palliative care professionals through the use of technology. , Telehealth now creates physical distance, functioning to ensure access while affording an infection-control barrier between patients and care providers and preserving protective equipment. , In recognizing the role of telehealth in the coronavirus pandemic and in the eventual postpandemic world, the experience of palliative care patients interfacing with telehealth warrants urgent and deliberate attentiveness.

After the pandemic, both care providers and patients may remain drawn to potentially attractive features of telehealth such as the ability to be seen at home, the ability to include family from other regions of the country in visits, and reductions in the need for physical office space. Patient and family caregivers deserve opportunities to provide feedback on whether telehealth encounters still maintain a sense of human connectedness and connectivity essential to the fields of palliative care and hospice care. An essential consideration in the goal of telehealth to extend care and communication reach with equity is to explore whether there has been inclusion of diverse populations for perspectives on telehealth experiences. If telehealth is to serve as a partnership tool, access and assessments of telehealth necessarily should reflect community input.12, 13, 14

Under the Coronavirus Preparedness and Response Supplemental Appropriations Act, the Centers for Medicare and Medicaid Services specified that hospice providers may provide routine home care as well as face-to-face encounters for recertification via telehealth. The Health and Human Services Office for Civil Rights “will exercise enforcement discretion and waive penalties for Health Insurance Portability and Accountability Act violations,” allowing care providers to communicate with patients and families via commonly used communication tools such as Skype and FaceTime. The Children's Health Insurance Program, private insurers covering pediatric cohorts, and Medicaid reduced barriers to telehealth use for child and family caregiver behavioral health assessment and management, symptom-based care, and concurrent wellness care for children. Palliative care teams are navigating rapid telehealth uptake while also attending to the patient, family caregiver, and interdisciplinary team communication experiences during an exponential increase in telehealth use in clinical practice.

A prior systematic review, aptly titled “telehealth in palliative care is being described but not evaluated,” recognized the lack of available evidence documenting perceptions about telehealth encounters for patients and family caregivers. To move toward a robust understanding of telehealth communication experiences, palliative care teams benefit from awareness of the survey instruments currently available to assess patient care experience using telehealth modalities. The objective of this review was to locate survey instruments used to assess telehealth interactions, to determine the content and constructs covered by the available instruments, and to describe the patient populations previously surveyed by the existing instruments.

Methods

Search Strategy

This review and its reporting followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, with the protocol registered in The International Prospective Register of Systematic Reviews as CRD42020200468. With the guidance of a medical research librarian, a search strategy was formulated using keywords and related subjects inclusive of (“telehealth,” OR “telemedicine”) AND (“survey(s)” OR “questionnaire”) AND (“experience” AND “communication”) AND (“validity” OR “reliability”). Three databases were searched (PubMed, Embase, Cumulative Index to Nursing and Allied Health Literature [CINAHL]) for articles matching the inclusion criteria (Appendix I).

Study Selection

Inclusion criteria included any article that was published in English between 2005 and 2020 which contained the development or analysis of a telehealth survey. The year 2005 marked a transition time between telephonic-based and video-based telehealth. Case report(s), editorial(s), letter(s), commentaries, and opinion documents were excluded.

Manuscripts were uploaded into EndNote for review of the potential inclusionary articles. The initial search generated 4887 citations with 7 citations hand-selected from prior study team familiarity with the literature. Duplicate studies were removed, and the remaining 3154 articles were screened in accordance to the established inclusionary criteria. Three authors assessed each manuscript for potential inclusion initially at the title/abstract level (J. L., B. W., M. S. W.) and then at the full-text level so that each article was independently assessed by a minimum of two of the three participating reviewers (J. L., B. W., M. W.). Interrater reliability was notably >85% across the reviewers at each stage of manuscript review (Figure 1 ).

Fig. 1

Prisma flow diagram.

Data Extraction

All authors abstracted data to a shared virtual spreadsheet fitted with the articles and outcomes, with a primary data extractor and a blinded data check partner assigned per survey. The data-extraction sheet was piloted on five surveys and revised based on study group discussion. Fields of data extraction interest included survey item numbers, question types, target population (health provider, patient, or both); languages; constructs; communication experience (perception regarding the human interaction) or communication modality (perception regarding the technology interface) question measures; theoretical frameworks used in survey development; and prior hospice or palliative care survey use. Validity and reliability were each reported as a binary variable (yes/no) within the data-extraction form with additional free text for description of how psychometric properties were reported.

Diversity Snapshot

To explore the populations previously included in telehealth survey research, PubMed was searched as a sole “snapshot” database using each survey name and acronym. Two study team members engaged in data extraction using a shared document designed and piloted by the study team (M. L. N., M. S. W.). The following items were obtained for each manuscript using the telehealth survey: patient or family caregiver or other respondent; pediatric or adolescent (defined as age <19 years), adult, or geriatric (defined as age >65 years) participants; geographic location; description of diagnoses; gender diversity; survey language; rural or urban setting; socioeconomic summaries; and ethnic/racial representation.

Results

Twelve telehealth communication assessment instruments were identified and agreed upon by the study team members. The constructs measured by the survey instruments and exemplary survey questions depicting the communication modality or human interaction themes are provided in Table 1 . Mean survey length was 20 questions (median 18, range 5-60 questions). Surveys consisted of Likert scale responses ranging from 3- to 11-point responses with a mean 5-point scale. One survey was inclusive of space for free-text responses complimenting the quantitative responses (Computer System Usability Questionnaire [CSUQ]), and one survey was qualitative (Telenursing Interaction and Satisfaction Questionnaire [TISQ]).

Table 1

Telehealth Survey Instrument Constructs, Content, and Psychometric Properties

Scale	Constructs	Exemplary Technology Modality Questions	Exemplary Human Interaction Questions	Psychometric Properties
Telehealth Satisfaction Scale (TeSS)	Quality, length of time to access, personal comfort, ease of use, privacy, attitude	“How satisfied were you with the voice quality of the equipment?”	“How satisfied were you with the thoroughness, carefulness and skillfulness of the health care team?”	Cronbach's alpha 0.9Items reported as valid based on eigenvalue >1.
Technology Acceptance Model (TAM)	Perceived usefulness, perceived ease of use, attitude, intention to use	“It was simple to use this system”; “The way I interact with this system is pleasant”	NA	Cronbach's alpha 0.91Items reported valid based on the high value of loading factor (>0.5).
Telemedicine Satisfaction and Usefulness Questionnaire (TSUQ)	Perceived usefulness, perceived effectiveness, perceived ease of use, attitude, intention to use, comparing telemedicine to in-person	“I can always trust the equipment to work”	"Provider engages me in my care"’; "Provider gets a good understanding of my concerns"	Cronbach's alpha 0.92Construct validity supported by a two-factor solution, Video Visits and Impact and Use, which explain 64% of variance.
Patient Assessment of Communication During Telemedicine (PACT)	Patient-centered communication, provider competence, interpersonal skills, convenience	NA	"How much did your doctor understand what you were going through emotionally?"; "Did you and your doctor decide together which of your concerns was most important to you?"	Cronbach's alpha 0.9Developed based on a previously validated patient satisfaction instrument.
Telemedicine Perception Questionnaire (TMPQ)	Communication, privacy/confidentiality, time and cost savings, difficulty, accessibility, physical contact, trust in equipment, standardization for future, satisfaction	“I trust this equipment to work;” “My privacy is maintained with this technology”	“"Clinician can get a good understanding of my medical problem"; "Can be as satisfied as if talking in person"	Cronbach's alpha 0.83Robust face, construct, and content validity processes reported.
Telehealth Usability Questionnaire (TUQ)	Usefulness, ease of use and learnability, interface quality, interaction quality, reliability and effectiveness, satisfaction	“The system is simple and easy to understand”; “I could hear the clinician clearly using the telehealth system”	“I think the visits provided over the telehealth system are the same as in-person visits”	Cronbach's alpha 0.8Due to development from prior validated scales, validity not reported.
Telemedicine Satisfaction Questionnaire (TSQ)	Satisfaction, technical quality, interpersonal manner, communication, financial aspects, time, accessibility and convenience	“I can hear my health-care provider clearly”	"Health-care provider is able to understand my health-care condition"; "I feel comfortable communicating with my health-care provider"	Cronbach's alpha 0.93Three components with eigenvalues > 1.0, which together explained 68% of the total variance.
System Usability Scale (SUS)	Usability primarily; also technical quality, ease of use, complexity of use, and user confidence	“I do not think that I would need the support of a technical person to be able to use this system”; “I found the various functions of this system well integrated”	NA	Cronbach's alpha 0.92Moderate convergent validity reported with other measures of usability.
Perceived Efficacy in Patient-Physician Interactions (PEPPI-5)	Efficacy primarily; confidence in ability to access care, make the most of visit, act upon conversation	NA	“How confident are you in your ability to ask questions? And, to get answers in this visit?”	Cronbach's alpha 0.92High structural and construct validity reported.
Patient Experience Questionnaire (PEQ)	Communication experience, emotions, short-term outcomes, barriers, relationship	NA	“The doctor understood what was on my mind”; “It was a bit difficult to connect with the doctor”	Cronbach's alpha 0.7Good construct validity described.
Computer System Usability Questionnaire (CSUQ)	System usability and capability	“The interface of this system is pleasant”; “This system has all the functions and capabilities I expect it to have”	NA	Cronbach's alpha 0.9Correlation validity 0.4; reasonable concurrent validity reported.
Telenursing Interaction and Satisfaction Questionnaire (TISQ)	Perceived interaction; inclusive of affective support, health information, decisional control, professional/technical competence) understanding, satisfaction	NA	“Did you perceive that you were given the opportunity to ask all your questions?”; “How satisfied were you with the nurse's ability to support you affectively?”	Cronbach's alpha 0.82Good content validity reported.

Three instruments (25%) inquired only into technology modality in terms of usability, connectivity factors, and the satisfaction with the video/audio interface (Technology Acceptance Model [TAM], System Usability Scale [SUS], CSUQ). Four instruments (33%) solely explored the experience of human interaction in terms of the perceived quality of communication, visit content, therapeutic relationship, or care experience (Patient Assessment of Communication During Telemedicine [PACT], Perceived Efficacy in Patient-Physician Interactions [PEPPI-5], Patient Experience Questionnaire, TISQ). Five instruments (42%) explored both the communication modality and the human experience with communication (Telehealth Satisfaction Scale [TeSS], Telemedicine Satisfaction and Usefulness Questionnaire [TSUQ], Telemedicine Perception Questionnaire [TMPQ], Telehealth Usability Questionnaire [TUQ], Telemedicine Satisfaction Questionnaire [TSQ]).

Survey instruments were notably unidirectional with the intention to assess a patient or patient family member's experience with telehealth rather than both provider and patient dyadic experience. Telehealth instruments could be applied to patient experience in 10 (83%) scales (TeSS, TSUQ, PACT, TMPQ, TUQ, TSQ, PEPPI-5, Patient Experience Questionnaire, CSUQ, TISQ). Only two (17%) scales were formatted for opportunity for both patient and provider comparative assessments of the telehealth encounter (TAM, SUS).

Theory was referenced in the development of three scales: Technology Acceptance Model (TAM), Telehealth Usability Questionnaire (TUQ), and the Telenursing Interaction and Satisfaction Questionnaire (TISQ).

Survey adaptation included the development of the TUQ scale created as a combination of questions from the TSW, TSUQ, and TMPQ scales. The original TAM was adapted into three survey editions (TAM3) relevant to telehealth.

Psychometric properties of the scales are reported in Table 1. Cronbach's alpha is notably high for the available telehealth assessment instruments with recognition of generally high internal consistency reported for survey items. Validity reporting was notably diverse with many survey instruments describing validity as “good” or “reasonable” or “high” but lacking quantified measures of validity (Table 1). Validity was not described for the two surveys developed from prior validated survey items (PACT, TUQ).

All survey instruments were validated in English with Chinese (TAM, TSQ), Dutch (TSQ, PEPPI-5), Spanish (TSUQ), and Swedish (TISQ) as additional available languages. Manuscripts lacked mention of use of translators or interpreters during telehealth encounters or telehealth assessments.

Specific to the field of palliative care or hospice care, telehealth survey instruments have been applied in assessing the transition of adult rural and pediatric patients to home hospice (TAM) and palliative and hospice care patient symptom reporting usage (SUS). These studies, notably with small sample sizes, depict telehealth as acceptable according to pediatric self-report, family caregiver insight, and adult cohorts based on the efficiency of the technology system and the effectiveness of the audio/video communication modalities. , Technology-fostered interactions were deemed useable communication prototypes for facilitation of communication, shared decision-making, and self-management by 18 surveyed adult palliative care patients. Telehealth as a care and communication modality was notably more acceptable to children receiving hospice and their family caregivers than to hospice nurses, warranting additional consideration of interdisciplinary team perspectives on telehealth.

Previous Populations

Forty-five articles were assessed for populations included in prior telehealth research. Table 2 provides a summary of respondent demographics according to telehealth survey instrument use. Telehealth survey instruments have primarily been used to assess patient experience (n = 41, 91%) , , 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64 with less frequent use to assess family caregiver (n = 7, 16%) , , , , , or medical provider experience (n = 8, 18%). , , , 67, 68, 69, 70, 71

Table 2

Summary of Participant Population Diversity by Technology Survey Instrument

Instrument	No. of Articles	Study Populationa			Study Age Groupa			No. of Articles in which Diversity Data Are Provided
		Patients	Caregivers	Providers	Adults	Children	Geriatric Participants Included?	Gender	Race/Ethnicity	SES	Urban/Rural
TeSS	3	3	3	0	4	1	1	2	0	1	4
TAM	10	9	2	2	9	1	6	9	2	5	6
TSUQ	2	2	0	0	2	0	2	2	1	0	1
PACT	1	1	0	0	1	0	0	1	1	1	1
TMPQ	1	1	0	0	1	0	1	1	0	0	1
TUQ	6	3	1	4	6	0	1	3	2	2	3
TSQ	2	2	0	0	2	0	1	2	0	1	0
SUS	13	12	0	1	13	0	0	12	3	5	1
PEPPI-5	3	2	1	0	3	0	2	3	2	3	1
PEQ	2	2	0	0	2	0	1	2	0	0	0
CSUQ	1	1	0	0	1	0	Unclear	0	0	0	0
TISQ	1	1	0	0	1	0	Unclear	0	0	0	0
Total	45	39	7	7	45	2	15	37	11	18	18

TeSS = Telehealth Satisfaction Scale; TAM = Technology Acceptance Model; TSUQ = Telemedicine Satisfaction and Usefulness Questionnaire; PACT = Patient Assessment of Communication During Telemedicine; TMPQ = Telemedicine Perception Questionnaire; TUQ = Telehealth Usability Questionnaire; TSQ = Telemedicine Satisfaction Questionnaire; SUS = System Usability Scale; PEPPI-5 = Perceived Efficacy in Patient-Physician Interactions; PEQ = Patient Experience Questionnaire; CSUQ = Computer System Usability Questionnaire; TISQ = Telenursing Interaction and Satisfaction Questionnaire; SES = socioeconomic status.

Could include more than one category.

Telehealth surveys have been underused to assess the experience of pediatric or adolescent cohorts with only two (4%) surveys used to assess the experience of pediatric care experiences or pediatric family caregiver experiences: TeSS for use in pediatric surgery telehealth communication and TAM for pediatric telehospice care. Fifteen studies (33%) included geriatric participants.

Demographic data for participants responding to the instruments are notably lacking in telehealth research articles with only 11 (24%) studies reporting on race or ethnicity, 18 (40%) reporting on some form of socioeconomic status, 18 (40%) reporting on geographic urban or rural representation, and 37 (82%) reporting on gender. For those articles reporting demographic data, most telehealth survey instruments respondents across telehealth survey scale options are noted to be Caucasian, with a postsecondary education, and not of the low-income category (Table 3 ).

Table 3

Summary of Diversity Inclusion by Survey Instrument

Instrument	Number of Articles	If Diversity Information Is Quantified, Percent Represented in Sample per Groupa
		Percent Female	Percent Nonwhite	Percent Low-Income	Percent HS Education or Below	Percent Rural
SUS	13	70%	30%	x	21%	x
TAM	10	64%	38%	62%	55%	x
TUQ	6	58%	8%	x	25%	x
TeSS	3	49%	x	x	53%	58%
PEPPI-5	3	69%	25%	x	40%	x
TSUQ	2	51%	46%	x	x	x
PEQ	2	64%	x	x	x	x
TSQ	2	x	x	x	x	x
PACT	1	4%	4%	16%	x	x
TMPQ	1	57%	x	x	x	x
TISQ	1	x	x	x	x	x
CSUQ	1	66%	x	x	87%	x

SUS = System Usability Scale; TAM = Technology Acceptance Model; TUQ = Telehealth Usability Questionnaire; TeSS = Telehealth Satisfaction Scale; PEPPI-5 = Perceived Efficacy in Patient-Physician Interactions; TSUQ = Telemedicine Satisfaction and Usefulness Questionnaire; PEQ = Patient Experience Questionnaire; TSQ = Telemedicine Satisfaction Questionnaire; PACT = Patient Assessment of Communication During Telemedicine; TMPQ = Telemedicine Perception Questionnaire; TISQ = Telenursing Interaction and Satisfaction Questionnaire; CSUQ = Computer System Usability Questionnaire.

Averaged across articles; categories defined by articles.

Discussion

Palliative care teams may consider use of survey instruments to assess patient experiences with telehealth. Palliative care and hospice research has focused extensively on descriptive methodologies to assess telehealth experiences. While telehealth has been deemed useable and acceptable in palliative care outreach, the methodology to determine such has primarily been qualitative inquiry and small-scale studies. Familiarity with the constructs assessed by available telehealth survey instruments is paramount in differentiating whether the focus is on the technology interface or the human interaction or both. The boundaries of the constructs do blur in certain question formats, such as in asking whether the participant experienced the telehealth interaction as equal to an in-person encounter. Not only do palliative care providers and researchers need to have familiarity with the constructs assessed but they also benefit from understanding the strengths and weaknesses of the different assessment tools, including whether these tools are reliable and valid.

Reasons for prior slow telehealth uptake in palliative care and hospice include patient discomfort with accessing or using technology for medical care, family caregivers’ concerns about the usability of technology for adequate support, concerns about the difficulty of technology use or reliability, and fear about jeopardizing the integrity of the provider-patient relationship with a screen barrier as compared to face-to-face care. Lack of access to smartphone or computer services and the internet platforms required for telehealth use has been understudied, raising concern about telehealth reach into underserved areas such as tribal homelands and rural regions. ,

In recognizing the impact of the coronavirus pandemic, there may be a norm-referenced shift in responses as survey questions regarding the “convenience” of telehealth as a communication modality are now confounded by not just time and distance saved but also perceptions of preventative public health measures. Even survey questions regarding “acceptability” are influenced by the now-normative approach to virtual interactions in nonmedical settings (schooling, extended family connectedness, community gatherings) as a public health safety measure. Palliative care researchers thus warrant caution in comparing responses to current telehealth surveys in relation to a preset, prepandemic standard.

With respect to the inclusion of diverse populations in the validation and utilization of the various telehealth assessment measures, our team discovered that what is not reported speaks just as loudly as what is. In fact, our review revealed a concerning lack of reporting on the demographic makeup of study participants in addition to a lack of diversity. Children and adolescents as well as geriatric age cohorts are notably underrepresented, and family caregivers are only rarely included in telehealth survey instrument reporting. Further, nonwhite or low-income populations and those without a postsecondary education are notably underrepresented in studies using the existing telehealth instruments. Educational exposures to varying technology interfaces, generational interpretation, and comfort with technology, culture, geography, economic access to personally owned technology, and language are sure to impact perceptions of telehealth. Furthermore, particularly with renewed calls for inclusion and the importance of listening to diverse voices in all facets of the public sphere, attentiveness to diversity and inclusion in eliciting feedback on telehealth experiences is critical. Lack of diversity reduces the generalizability of the existing telehealth survey findings.

Limitations of this study include the restriction to English-language manuscripts. The limited number of research-based telehealth manuscripts and the lower quality of the available studies further limit overall application of findings.

Conclusion

Logical next steps in telehealth research in palliative care include the development and then use of validated telehealth experience survey instruments with paired consideration of not only technology interface but also measure of human interaction. The uniqueness of palliative care practice makes the existing instruments helpful but likely insufficient for direct application. Ideally future telehealth research would include triangulated patient, family caregiver, and provider scale use for comparison in perspectives of the same virtual encounters. Future telehealth research for patients receiving palliative care may consider methods that concurrently and longitudinally assess symptom burden, quality of life, and cost analyses.

Overall, this review revealed that instruments exist to measure patient and family caregiver perceptions of telehealth, but the instruments are currently underused. Caution should be taken to ensure telehealth support meets the cultural and educational needs of the communities that have not yet provided perspective on telehealth encounters. The lack of participant diversity in reporting on telehealth communication using telehealth survey instruments warrants attentiveness to inclusion to better understand perspectives on telehealth encounters.