Patient perspectives of telemedicine quality.

BACKGROUND: The purpose of this study was to explore the quality attributes required for effective telemedicine encounters from the perspective of the patient.
METHODS: We used a multi-method (direct observation, focus groups, survey) field study to collect data from patients who had experienced telemedicine encounters. Multi-perspectives (researcher and provider) were used to interpret a rich set of data from both a research and practice perspective.
RESULTS: The result of this field study is a taxonomy of quality attributes for telemedicine service encounters that prioritizes the attributes from the patient perspective. We identify opportunities to control the level of quality for each attribute (ie, who is responsible for control of each attribute and when control can be exerted in relation to the encounter process). This analysis reveals that many quality attributes are in the hands of various stakeholders, and all attributes can be addressed proactively to some degree before the encounter begins.
CONCLUSION: Identification of the quality attributes important to a telemedicine encounter from a patient perspective enables one to better design telemedicine encounters. This preliminary work not only identifies such attributes, but also ascertains who is best able to address quality issues prior to an encounter. For practitioners, explicit representation of the quality attributes of technology-based systems and processes and insight on controlling key attributes are essential to implementation, utilization, management, and common understanding.

Telemedicine involves the use of modern information technology, especially two-way interactive audio/video communications, computers, and telemetry, to deliver health services to remote patients and to facilitate information exchange between primary care physicians and specialists at some distances from each other.1

Introduction

As an integral component of telemedicine, high bandwidth video conferencing (also known as point-to-point connections) is used in medical education, peer consultation, and in direct patient care (see the American Telemedicine Association site at http://www.americantelemed.org for further details). This research limits its scope to telemedicine encounters that use medical video conferencing deployed for direct patient care.

There is widespread interest in utilizing this technology as an economical method to provide expert medical service to patients in remote and awkward locations and to address misdistribution of health care resources (ie, facilities and practitioners).1–3 However, utilization rates of installed telemedicine projects have not met expectations.4 While most previous work has found telemedicine to be medically effective, acceptance and use of telemedicine has not been well understood.5,6 Medical state licensure and cost reimbursement have been identified as barriers to utilization of installed systems, but it appears these impediments may not be long standing.7 Furthermore, mixed results have been reported in contexts where policy constraints were not a major impediment.8–13 These mixed results suggest that research must look more closely at telemedicine systems to understand how to improve utilization rates.

One reason for low utilization rates may be dissatisfaction with the telemedicine encounter experience. Early work looked at this question from the point of view of the physician,14 but few studies have looked at it from the patient perspective. Most prior patient satisfaction studies suggest that teleconsultation is acceptable to patients. Yet, this work has not provided adequate insights into what drives patient satisfaction levels with telemedicine;15 instead it is mostly technology-centered and focused on utilization rates. Studies on telemedicine quality to date tend to fall into two categories. The first category addresses quality in terms of net benefits, namely the quality of patient care or stakeholder satisfaction,15–20 with limited insight regarding quality antecedents (attributes). The second category focuses on specific technology quality attributes, such as the quality of audio and/or video aspects of the technology.21–23 In spite of increasing use and interest in telemedicine, generalized standards of quality that encompass the patient consultation experience have not emerged. Studying telemedicine quality from the patient perspective, as consumer and indirect user, is needed from a health care business perspective,24 as health care has become a competitive industry where patient opinions are shaping the marketplace and may be missed by an organization or provider if not directly studied.

The purpose of our research is to propose a patient-oriented taxonomy of telemedicine service quality attributes for direct patient care using high bandwidth video conferencing (hereafter referred to as medical video conferencing). We focus on the attributes related to process quality as a means to understand service operations rather than quality of results (ie, service reliability; see Harvey for further discussion of distinctions).25 To facilitate the usefulness of this taxonomy, we also address issues of relevance and control for each attribute identified. We use a multi-dimensional, sociotechnical approach to understanding quality using multiple forms of data collection and multiple perspectives of analyses.

We address the following research questions:

Question 1 What quality attributes from the patient perspective contribute to telemedicine encounter success?

Question 2 Which telemedicine encounter quality attributes from the perspective of patients are considered most relevant to encounter success?

Question 3 At what point in time in relation to the encounter (eg, before, introduction, core, closure, after) are identified attributes most controllable?

Question 4 What entities (eg, organization, equipment manufacturer, doctor, patient) have the most control over each identified attribute?

Materials and methods

We conducted an institutional review board-approved multi-method, inductive field study. Qualitative methods (eg, focus group and interview) were used to elicit, code, and analyze data from respondents. Quantitative methods (eg, survey) were used to validate the analyses. We also used numerous sources (telemedicine practitioners and academic researchers) to code and interpret the data and to address triangulation in analysis. The logic involved in the development of the telemedicine taxonomy may be traced both from the participant data to the quality attributes and from the quality attributes to the participant data.

Site

The Department of Veterans Affairs (VA) VISN 8 was the site of this work. VISN 8 is one of the most active VA districts in the area of telemedicine, with a geographic coverage including parts of Georgia, FL, USA, and the Caribbean. We collected data from ten sites in VISN 8 located across the VISN 8 footprint that were identified as particularly active telemedicine spoke sites (patient side for service). The VA’s VISN 8 health system currently provides a diverse spectrum of medical specialties through video conferencing, clinic histories (newly established sites and ongoing sites), types of facility (strip mall clinics to hospital settings), and provider telemedicine experience (new to telemedicine and seasoned). Diversity among these factors was important to the nature of the study. Although we felt it important to introduce various telemedicine encounter experiences into the study, we chose not to introduce organizational diversity to avoid organizational noise.

Establishing the DeLone and McLean model as a contextual fit for telemedicine

By reviewing quality, telemedicine, marketing, service-oriented management and information systems literature as well as unstructured interviews with domain researchers, we developed the initial quality attribute framework. The starting point of our framework was based on constructs in the DeLone and McLean Information Technology quality model (comprised of service quality, system quality, information quality, use, user satisfaction, individual impact, and organizational impact).26 As noted by DeLone and McLean, as well as other domain experts, to make this framework usable in a particular environment, the specific criteria for quality must emerge from investigating the context of its use and understanding of the concerns of the stakeholder groups.27 To contextualize the DeLone and McLean model for telemedicine encounter quality, the following methods of data collection were used:

Direct observation (40 hours) of medical video conferencing rooms, functional equipment, and segments of video conference sessions for representations of quality attributes and issues

Review of archived video and photographic images of telemedicine encounters and rooms

Open-ended patient survey of 84 telemedicine patients (see Figure S1)

Unstructured interviews with an originator of the model as well as telemedicine researchers inside and outside of the USA.

In assessing the results of the patient survey questions that compared medical video conferencing with an in-person examination (see Figure S1), it appears that patients within our sample were, by and large, satisfied with their telemedicine encounter.28 However, it became apparent that the use of telemedicine system terminology familiar to patients (eg, physical environment; way the examination was conducted) rather than imposing new constructs (eg, support quality or use quality) was necessary in further discussion with the patients. All dimensions of the research framework (information quality, use quality, technology quality, and service quality) were referenced in some form during direct observation, interviewing, and through the patient survey. For example, patient survey responses to the best and worst aspects of the telemedicine encounter indicated such things as the physical environment, scheduling support, personal attention, and technology were important. The following are the refined definitions of the dimensions of functional quality reflecting both the stakeholder concerns and assumptions that we use to inspire and organize inquiry into specific telemedicine encounter quality attributes.

System (technology) quality: the features of medical video conferencing equipment and telecommunication processes utilized for medical video conferencing encounters.

Information quality: the characteristics of information that allow participants to take appropriate action concerning patient care and facilitate diagnosis. Telemedicine information quality attributes should include attributes that facilitate simultaneously capturing appropriate input for collaborative communication (eg, aspects of the physical environment) as well as providing appropriate technology transmission output.

Support (service) quality: the support provided for use of a telemedicine system during the encounter. In a telemedicine system, it can be defined as the human infrastructure and physical environment provided by the organization that support user comfort and system use.

Use quality: the informed and effective communication and deployment of technology by direct users (medical staff) during the medical video conferencing encounter that facilitates desired outcomes.

We adapted the constructs in the information systems success model26 used to guide this study as follows to align with the quality focus of this work:

Information quality

System (technology) quality

Service (support) quality

Use (quality).

This framework provides only a precursory understanding of the quality construct. There is no universal set of quality attributes for any of the suggested dimensions because quality is both multi-faceted and domain-specific.25,29

Focus groups to identify framework attributes

Once the preliminary framework was contextualized, we began to collect patient data to better understand what domain-specific quality attributes patients perceived as important to a telemedicine encounter. Patient encounters may be sporadic or limited (eg, one to two occurrences) precluding the development of expertise. Thus, we chose to use focus groups to derive expertise for a collective patient assessment of quality in the telemedicine encounter process.30

To assemble these groups, all patients involved in telemedicine encounters in the previous 3 months at participating facilities were identified. Three months was selected as a maximum time lag from an encounter experience as significant time lags may deteriorate recall to the point where the patient may make little contribution to the group effort. We randomly selected individuals from the 3-month list. VA employees solicited focus group participation via telephone. Prospects on the list were contacted until all planned groups were booked with six to seven participants (to address no-shows). Reminder letters and telephone calls were directed to all potential participants. Of the 32 patients agreeing to participate, 30 actually participated in the focus group process (94%). Focus groups took place at the facility where the patient had his/her medical video conferencing encounter(s) to facilitate recall.

Six focus groups were conducted with an average of six participants per group. Group members were mostly male. We balanced the diversity of medical video conferencing experiences (eg, patients seen by various doctors, in various facilities, by various specialists, and for various conditions) among our focus group participants. All focus group sessions were audio recorded and later transcribed.

Two members of the research team used an open, hermeneutic process to identify and define quality attributes based on information gathered from the focus group.31 We used NVivo (a software package that supports qualitative coding and analysis) to perform this coding. Coding consisted of specifying slips,32 listing attributes, and providing definitions of each attribute. The relevant literature provided insight into identifying terms and labels that have been associated with quality to provide inspiration for coding and to juxtapose emerging knowledge from this study with existing theory and knowledge.31,33

Two telemedicine service providers also coded all of the focus group transcripts (in parallel to the research team efforts) using an open, hermeneutic process. The objective of this “provider” coding was to provide interpretive contextual enrichment to the analysis (enhance relevance) and to determine conceptual convergence with researcher coding (enhance validity).32 These coders did not have a predefined coding schema and were encouraged to annotate their own insights in the coding process. The providers worked as a team to identify attributes and develop definitions. Parallel form procedures were used to assess correspondence among meanings of the attributes identified by practitioners and researchers through open coding.

To assess the reliability of the resulting attributes and their definitions, a third researcher (not engaged in prior open coding) dual-coded a random 33% stratum from each focus group and interview transcript, using the previously established attributes and definitions provided as a coding schema (interrater reliability). The stability of codes over time (intrarater reliability) was assessed by having the same coder code the interviews at two different time periods (second time, four months after the first effort).

Framework validation

To validate the framework that emerged from the focus groups, focus group members were contacted and asked to complete a paper-based survey that included both quantitative and qualitative questions, meant to assess the participants’ perspectives of the correctness, completeness, and relevance of model attributes (see Figure S2 for open-ended questions). The participants also assessed the importance of the proposed attributes.

Assessing attribute control – who and when

Recognition of the relative importance among attributes is needed for research and practice to most effectively utilize the model and focus attention. Additionally, an understanding of the relative importance of the attributes can facilitate the creation and interpretation of formative measures for each dimension of telemedicine quality.

The usefulness of a model of telemedicine service encounter quality attributes to both practice and research is greatly enhanced when there is some understanding of when in the encounter certain attributes of quality can be manipulated and who can manipulate them. To further facilitate application and understanding of the field study findings, we assembled an expert panel of two providers and four researchers, possessing complementary expertise. The six-member panel independently completed a paper-based survey responding to the following two questions for each of the identified attributes:

When is the quality attribute most controllable?

Before encounter

Beginning of encounter

Body of encounter

Encounter closure

After encounter

Not controllable

Who has most control over the quality attribute?

Consulting physician

Telemedicine coordinator

Patient

Scheduler

Technical support

Equipment developer

Medical center management

External environment

The panel members were instructed to use their personal knowledge as well as their recall of the coded focus group transcripts to answer each question. In the spirit of comprehensive model development, we allowed participants to select multiple attributes in response to each question, as they deemed appropriate. The independent responses were aggregated in a table with differences highlighted and sent to each panel member for review. Panel members met to collectively discuss the differences noted in the aggregated table, acknowledging various insights and clarifying attribute implications. These discussions were followed by additional rounds of survey/review until consensus was reached.

Results

Contextualized framework of quality attributes

We present the resulting contextualized framework and quality attributes for the medical video conference derived from the focus group and validating procedures in Figure 1. Our framework uses four high-level categories of quality (system/technology quality; information [input/output] quality; service/support quality; and use [informed/effective] quality) supported by foundational work done to establish the DeLone and McLean model as a contextual fit for telemedicine (see section entitled “Establishing the DeLone and McLean model as a contextual fit for telemedicine”). To facilitate research rigor and appropriate representation in categorizing the identified attributes within framework constructs, three researchers familiar with this study, the coding process, and the DeLone and McLean model, participated in the mapping process. The researchers first performed an independent paper-based mapping, working from a list of attributes and definitions, and then participated in two rounds of meetings to collectively reach consensus. Aside from providing an appropriate representation of our research model, this process was employed to address any potential of either identified higher-order constructs being inappropriate to this domain or the need for additional higher-order quality constructs. No modification to the higher-order constructs was deemed necessary. However, the researchers subdivided information quality attributes into technology and physical environment aspects and subdivided support quality attributes into human and physical environment dimensions to enhance model precision. The identification of attributes across all quality dimensions indicates the propriety of the research framework as a foundation for the patient perspective.

Figure 1

Telemedicine service encounter quality model – patient perspective.

Regarding the identification of specific attributes under each category, several rounds of parallel form procedures were used to assess correspondence among meanings of the attributes identified by practitioners and researchers through open coding. Common themes were found throughout the process and labels for the attributes were derived through consensus (see Table S1 for final definitions). Final consensus resulted in six to eight specific quality attributes (eg, reliability, audio clarity, technical support) derived from patient focus groups for each of the four high-level categories of quality.

Results of dual coding by a third researcher using the attribute model and definitions previously reconciled indicated a high interrater reliability (consistently >90%). The “recoding over the passage of time” intrarater reliability measurement technique resulted in high intrarater reliability (consistently >95%). No patterns were noted in the intrarater or interrater differences that might reflect the need to modify the attribute listing.

The participants’ assessment of the importance of all proposed attributes in the framework had a mean in excess of three (anchor point, important) on a four-point scale (four anchor point, critical), indicating that the participants considered all attributes to be relevant. Given the opportunity, participating patients did not mention any missed attributes. Table 1 indicates the relevancy scores of each of the quality attributes. The presentation order of the constructs in Table 1 is an indication of the importance of each attribute (high numbers indicate most relevant attributes) based upon Likert-type scale measures from the survey administered to the focus group for validating purposes. The scores are ranked in order by highest mean, then lowest standard deviation.

Table 1

Results of patient validation and relevancy score

Response scale label	Not important	Slighty important	Important	Highly important	Not answered	Mean	SD
Weight	1(Count%)	2(Count%)	3(Count%)	4(Count%)	5(Count%)
Clear future directives	1 (5%)	0 (0%)	2 (10%)	18 (86%)	0 (0%)	3.762	0.700
Audio clarity	0 (0%)	0 (0%)	6 (29%)	15 (71%)	0 (0%)	3.714	0.463
Telemedicine-trained staff	0 (0%)	0 (0%)	8 (38%)	13 (62%)	0 (0%)	3.619	0.498
Peripheral sophistication	1 (5%)	0 (0%)	5 (24%)	15 (71%)	0 (0%)	3.619	0.740
Medical team coordination	0 (0%)	0 (0%)	9 (43%)	12 (57%)	0 (0%)	3.571	0.507
Image resolution	0 (0%)	1 (5%)	7 (33%)	13 (62%)	0 (0%)	3.571	0.598
Professionalism – clinician in patient room	0 (0%)	0 (0%)	10 (48%)	11 (52%)	0 (0%)	3.524	0.512
Patient education/orientation	0 (0%)	0 (0%)	10 (48%)	11 (52%)	0 (0%)	3.524	0.512
Conveys access/review of medical records	0 (0%)	1 (5%)	8 (38%)	12 (57%)	0 (0%)	3.524	0.602
Consultant telepresence	0 (0%)	0 (0%)	11 (52%)	10 (48%)	0 (0%)	3.476	0.512
Adequate lighting	0 (0%)	2 (10%)	8 (38%)	11 (52%)	0 (0%)	3.429	0.676
Scheduling support	0 (0%)	0 (0%)	13 (62%)	8 (38%)	0 (0%)	3.381	0.498
Technical support	0 (0%)	1 (5%)	11 (52%)	9 (43%)	0 (0%)	3.381	0.590
Reliability	1 (5%)	2 (10%)	6 (29%)	12 (57%)	0 (0%)	3.381	0.865
Focus on patient care	0 (0%)	0 (0%)	14 (67%)	7 (33%)	0 (0%)	3.333	0.483
Privacy	0 (0%)	2 (10%)	11 (52%)	8 (38%)	0 (0%)	3.286	0.644
Usefulness	1 (5%)	0 (0%)	12 (57%)	8 (38%)	0 (0%)	3.286	0.717
Affordability	1 (5%)	1 (5%)	9 (43%)	8 (38%)	2 (10%)	3.263	0.806
Quiet/soundproof	0 (0%)	5 (24%)	7 (33%)	9 (43%)	0 (0%)	3.190	0.814
Motion handling	0 (0%)	3 (14%)	13 (62%)	5 (24%)	0 (0%)	3.095	0.625
Mix with in-person exams	0 (0%)	2 (10%)	15 (71%)	3 (14%)	1 (5%)	3.050	0.510
Ergonomic design	0 (0%)	5 (24%)	11 (52%)	5 (24%)	0 (0%)	3.000	0.707
System feedback to patient	3 (14%)	6 (29%)	4 (19%)	8 (38%)	0 (0%)	2.810	1.123
Suitable temperature	1 (5%)	6 (29%)	14 (67%)	0 (0%)	0 (0%)	2.619	0.590
Facilitating décor	3 (14%)	6 (29%)	11 (52%)	1 (5%)	0 (0%)	2.476	0.814
Performance	5 (24%)	6 (29%)	7 (33%)	3 (14%)	0 (0%)	2.381	1.024
Total	17 (3%)	49 (9%)	242 (44%)	235 (43%)	3 (1%)

Abbreviation: SD, standard deviation.

For the most part, physical environment attributes (eg, facilitating décor and suitable temperature) seem to be on the lower end (under three points on a four-point scale). Scores above 3.5 on the four-point scale indicate that “basic” technical functionality (audio clarity, image resolution, and peripheral sophistication) coupled with aspects of the communication process (clear future directives, medical team coordination, telemedicine trained staff, professionalism, patient education/orientation, conveying access, and review of medical records) are most critical in this context.

Control of attributes – when and who

After two iterations of independent mapping and sharing results, the expert panel collectively reached consensus regarding when each quality attribute was most controllable and who has the most control over the quality attribute. When these quality attributes can be controlled is shown in Table 2 and who can control them is shown in Table 3. Table 2 illustrates that the majority of the attributes can be controlled in multiple time frames associated with the encounter, and almost all have the ability to be controlled prior to an encounter beginning. Table 3 indicates that most of the quality attributes are not under the control of the patient but rather under the control of either medical providers, those responsible for the telemedicine equipment used during the encounter, or various hospital administrators (including telemedicine coordinators and schedulers).

Table 2

When quality attribute is most controllable for telemedicine service encounters

Quality attributes/encounter control points	Before the encounter	Encounter start	Encounter body	Encounter end	After the encounter
System quality attributes
Reliability	⊕		⊕
Peripheral sophistication	⊕
Ergonomic design	⊕
Performance	⊕		⊕
Usefulness	⊕		⊕
Affordability	⊕
Information quality attributes
System feedback to patient	⊕	⊕	⊕
Quiet/soundproof	⊕
Adequate lighting	⊕		⊕
Audio clarity	⊕	⊕
Motion handling	⊕		⊕
Image resolution	⊕		⊕
Service quality attributes
Technical support	⊕	⊕	⊕	⊕
Scheduling support	⊕				⊕
Patient education	⊕	⊕
Privacy	⊕	⊕	⊕	⊕
Suitable temperature	⊕	⊕	⊕
Facilitating décor	⊕
Use quality attributes
Focus on patient care	⊕
Consultant telepresence	⊕	⊕	⊕	⊕
Medical team coordination	⊕	⊕	⊕	⊕
Telemedicine trained staff	⊕
Conveys access/review of patient records		⊕	⊕
Clear future directives				⊕	⊕
Professionalism (clinician in patient room)	⊕	⊕	⊕	⊕
Mix with in-person examinations	⊕

Note: ⊕ indicates when each attribute is most in control for each medical video conferencing quality attribute.

Table 3

Role most in control for medical video conferencing quality attribute

Quality attributes/controller roles	Provider roles		Consumer role	Technical roles		Organizational roles			External
	Physician	In-room clinician	Patient	Equipment manufacturer	Technical support	Management	Telemedicine coordinator	Scheduler	Environment
System quality attributes
Reliability				⊕	⊕				⊕
Peripheral sophistication				⊕
Ergonomic design				⊕
Performance	⊕			⊕
Usefulness	⊕	⊕		⊕	⊕		⊕
Affordability				⊕	⊕	⊕	⊕		⊕
Information quality attributes
Quiet/soundproof						⊕	⊕		⊕
Adequate lighting	⊕	⊕				⊕	⊕
System feedback to patient	⊕	⊕		⊕			⊕
Audio clarity	⊕	⊕		⊕	⊕
Motion handling	⊕			⊕	⊕
Image resolution	⊕			⊕	⊕
Service quality attributes
Technical support					⊕		⊕
Scheduling support	⊕	⊕	⊕					⊕
Patient education		⊕	⊕				⊕	⊕
Privacy	⊕	⊕				⊕	⊕
Suitable temperature		⊕			⊕		⊕
Facilitating décor	⊕	⊕				⊕	⊕
Use quality attributes
Focus on patient care	⊕			⊕	⊕		⊕
Consultant telepresence	⊕	⊕
Telemedicine trained staff	⊕	⊕			⊕		⊕
Medical team coordination	⊕	⊕					⊕
Conveys access/review of patient records	⊕
Clear future directives	⊕	⊕	⊕
Professionalism (clinician in patient room)	⊕	⊕
Mix with in-person examinations	⊕		⊕			⊕	⊕	⊕

Note: ⊕ indicates what role is most in control for each medical video conferencing quality attribute.

Discussion

Attributes of a quality telemedicine encounter

Organizations will only achieve advantage through quality when there is a match between the importance that consumers (patients) assign to individual quality attributes and the organization’s performance along those dimensions.34 While 89% of the patients felt the medical video conference examination was the same or better than an in-person examination, 42% felt that their experience could be improved (see Figure S1). The identification of attributes across all quality dimensions (use quality, service quality, system quality, and information quality) implies that both groups associate telemedicine quality with both social and technical factors. The intermingling of human (eg, technical support, scheduling support) and technical (eg, peripheral sophistication, audio clarity) attributes seems to exist throughout the ranking of relevance scores.

The quality attributes that the patients ranked higher (mean of greater than 3.5 as presented in Table 1) reflect what the patients perceive as critical components in their telemedicine service encounters. Namely, patients want instructions and guidance regarding their health provided via a means that addresses the challenges of distance and technology and that supports the information-sharing process. It is notable that peripheral sophistication, which addresses enhancing features, was ranked high (above 3.5, see Table 1) among the technology attributes list. Informal interviews with providers and direct observations of telemedicine examinations conducted during research groundwork indicate a provider may envision the ability to extend the capabilities of available equipment through some adaptation before peripherals are necessary. Patients may not have this vision and may feel more comfortable with equipment designated for a specific task. Given that physical environment attributes were among the lower-ranked attribute scores, it seems patients recognize comfort is desirable but may be willing to sacrifice some comfort to gain the conveniences of telemedicine.

Patients indicate that their medical provider should be able to focus on patient care, rather than figuring out technology. Technology should not get in the way of either the physician or the patient. To enable this focus, technical roles, organizational roles, and external factors must support provider efforts during windows of opportunity. Ideally, telemedicine will become another tool in the medical kit that can provide remote care where both the patient and provider walk away from the encounter with a feeling of fulfillment and success.

During a telemedicine encounter the consulting doctor is usually the person in charge. However, we found that many of the quality attributes are in the control of other actors and all of the attributes can (and should) be addressed in some way prior to an encounter. For an encounter to be successful, planning and preparation are essential. Technical, organizational, and medical provider roles must all work as a team and act before the encounter begins to maximize the potential for success.

The technical actions that appear to be most important to a successful telemedicine encounter are ones associated with the actual design of the equipment. Both technology quality attributes and information quality attributes are controlled by the equipment design. The equipment manufacturers need to ensure that the equipment is reliable, easy to maneuver (ergonomic), and can perform a range of medical tasks (peripheral sophistication).

The organizational actions that improve the successfulness of a telemedicine encounter can be largely managed prior to an encounter. These actions include managerial planning (in the form of staff training and patient information management) and building management (in the form of room design, ie, adequate privacy and lighting as well as room décor). Once the encounter begins, the provider and patient have a more active role in ensuring the quality level of the telemedicine encounter. With the help of some support roles, quality attributes, such as lighting, noise level, privacy, and technology performance, can be controlled. We also see communication-related attributes playing a strong role during the encounter. Style of communication (eg, professionalism-clinician in the room, “telepresence”) as well as communication content (eg, conveys access/review of patient records, clear future directives) are necessary. These attributes seem to have some element of technological influence. For instance, to control “telepresence”, both interpersonal skills and camera placement to create the image of virtual “eye contact” are needed.

Impact on practice

This research has led to a revision of training programs and telemedicine encounter protocols in the site locations that participated in the study. Additionally, patient telemedicine training and information brochures developed using data from this study are being used throughout the country by two major health care agencies. The model has also been used in the inception of new programs and as a tool for indoctrinating stakeholders in the telemedicine process. Furthermore, one telemedicine administration indicated the model served a need in the health care industry for a communal model that could be referenced by all telemedicine stakeholders. To this end, a telemedicine committee used the model during a planning session to provide a common language and framework from which to elicit current deficiencies in the existing telemedicine process. Results from the control and importance tasks in this study provided guidance to the committee regarding future action by identifying the process stage most affected by reported deficiencies and entities that could work toward remedies.

Limitations

This study took place within VISN 8 of the VA system in the USA. It is not clear if the findings are generalizable to private sector health facilities, or to other regions of the USA or within other countries. As the health and technological environment evolves, new attributes will likely arise and some attributes may become unworthy of mention. Furthermore, other health care delivery systems (telephone consults, mobile applications, in-home long-term care) may require adaptation of the quality attributes resulting from this study. Future research is necessary to understand the applicable boundaries of this framework.

Conclusion and future directions

The principal contributions of this paper are an organized taxonomy of quality attributes for a medical video conferencing system that contribute to encounter success from the patient perspective and indications of how these attributes can be controlled and by whom. Through this taxonomy, quality can then be assessed according to the presence or absence of such attributes and the fitness for their proposed use.35,36 The need to recognize both social and technical attributes as antecedents to success in the study of telemedicine service encounters is not a superficial one.

Attempting to define quality with enough precision to substantially enrich understanding and promote positive action for the sophisticated phenomenon of telemedicine service encounters is a complex task that begets our multi-method research. We employed an inductive field study as a means to define the elements of service quality that are vital to the success of a specific form of telemedicine encounter. We would direct researchers interested in studying quality to consider all relevant contextual attributes within the scope of the problem of interest (particularly those of critical importance).

We provide a second extension to understanding by identifying opportunities for control of each attribute. This effort enables researchers to better understand the encounter process from the patient perspective by associating attributes with the sequential phases of the encounter and with entities that may serve as points of control. Future research could focus on how the control of the quality of telemedicine service encounters is best achieved, given the multiple, disparate sources and points of control. Practices may use this model as a means to recognize and assess telemedicine quality standards, contemplate requirements while balancing constraints, and enact control directives. Retrofitting quality improvements into an existing system can be cost-prohibitive and technically challenging. Ideally, the design of a telemedicine system would begin with unambiguously stated quality requirements in conjunction with functional requirements. Our analysis of control issues indicates that there exist opportunities to economically and efficiently correct some attribute deficiencies beyond the planning stage (eg, patient orientation and adaptability). Telemedicine does introduce change, namely the utilization of technology service encounters, into the traditional process of health care delivery. The challenge to the service provider is to orchestrate the encounter so that all technical imperatives (and clinical needs, in this case) are met, while managing the social/communication process and customer expectations and perceptions. An understanding of the factors of quality and opportunities for control can help the change process as well as the ongoing management of telemedicine.

Patient concerns and issues survey questions and response summaries.

Abbreviations: SA, strongly agree; A, agree; D, disagree; SD, strongly disagree.

Open-ended questions on patient validating survey.

Table S1

Telemedicine quality attributes and definitions

Research team attribute	Practitioner team attribute	Definition
Technology (system) quality
Reliability	Reliability	The degree to which medical video conferencing equipment consistently connects and functions as designed.
Usefulness	Provider acceptance also part of usefulness (effective)	The degree to which, in general, stakeholders believe telemedicine devices function as intended and provide features (eg, camera freeze frame capability, wireless remote controls for cameras, and camera controls at the consultation site) that contribute to patient care in a clinical environment. This is a general perception that the equipment is effective for patient care and appropriate to the medical context.
Peripheral sophistication	Not detailed in equipment design	The degree to which technology that allows the extension of basic medical video conferencing equipment to specialized medical situations is available. Peripheral devices include electronic stethoscope, dermatology camera, retinal camera, nasopharyngoscope, and electronic electrocardiogram.
Performance	Not detailed in equipment design	The degree to which the telemedicine technology provides timely response to external stimuli to minimize delayed transmissions (eg, codec responsiveness to provide real-time delivery). Additionally, the speed of cameras to auto-focus and auto-iris (light adjustment) to suitable levels of image quality and the responsiveness of other input devices (eg, touch pads).
Ergonomic design	Not detailed in equipment design	The degree to which the technology is designed in a manner that affords ease in maneuvering and navigating medical video conferencing system equipment. Refers to the physical design of the equipment (eg, not bulky, accommodates hand span, not uncomfortable to use).
Affordability	Part of usefulness (efficiency)	The degree to which medical video conferencing provides a net positive relationship between the functional benefits of medical video conferencing and the resources (time, money, and space) consumed by the effort. Affordability issues include equipment costs, transmission costs, and patient reimbursement.
Information quality dimension
Audio clarity	Audio quality	The degree to which the technology provides sound quality comparable to similar communications occurring with all parties in one location. The distinctness of sound is compromised by choppy or muffled sound.
Image resolution	Image resolution	The degree to which the technology provides image resolution fidelity and precision for each still image. Image resolution quality is determined by the resolution of the camera receiving the image and the monitor projecting the image.
System feedback to patient	Appears to be part of equipment design definition	The degree to which the patient receives indicators from the technology that patient information was successfully received by the consulting doctor (eg, picture in picture on monitor feature used).
Motion handling	Motion handling	The degree to which the technology provides image refreshing at a rate that depicts comparatively real-time movement. Motion handling involves a smooth transition as images are updated. Motion handling quality is determined by the amount of pixelation, motion artifacts, and frame dropping that occurs during an encounter.
Adequate lighting	Lighting	The degree to which the amount and direction of lighting is appropriate to illuminating the patient examined and the physician at the consulting site without causing inappropriate shadows or discoloration.
Quiet/soundproof	Part of room design	The degree to which the acoustical environment provides sound control (ie, minimal noise interference) to improve both audio clarity and privacy in both the exam room and physician consulting room.
Support (service) quality
Scheduling support	Scheduling	The degree to which interactions between the scheduler and the multiple stakeholders facilitate integrating the schedules of multiple stakeholders (eg, patients, clinicians, technical staff). This implies the scheduler strives for efficiency in setting sequential appointments for clinicians. It also implies that the patient is apprised that they will have a telemedicine rather than in in-person encounter.
Patient education/telemedicine orientation	Orientation	The degree to which preliminary interaction occurs between the patient and a medical representative with the goal of facilitating patient comfort and understanding.
Privacy	Part of room design	The degree to which the exam room and physician consulting room are inaccessible to walk-in intrusion and visual or audible intrusion from walk-by traffic.
Technical support	Technical support	The degree to which interactions between medical video conferencing equipment operators and technology support personnel allow offloading technical issues from the clinician(s) to technology workers.
Suitable temperature	Part of room design	The degree to which the temperature accommodates both the equipment and patient comfort.
Facilitating décor	Room design	The degree to which the furnishings and other aspects of décor facilitate participant comfort in using the technology and in communicating during the encounter.
Use quality
Accommodates focus on primary objectives (patient care)	Not specifically addressed, but acknowledged the slips in this attribute were appropriate	The degree to which the technology permits the medical care provider to focus on patient care, rather than figuring out technology. The technology should not distract the provider (eg, doctor) and the consumer (eg, patient) from the tasks necessary to fulfill the objectives of the encounter. In essence, the equipment is subordinate to the exam process; it is merely a tool, not the focus.
Consultant telepresence	Telepresence	The degree to which the remote consultant is able to convey a ubiquitous, virtual presence (eg, eye contact, warm atmosphere, congenial atmosphere, interactive atmosphere). Telepresence may be thought of as “virtual bedside manner”.
Trained staff	Technical training	The degree to which users are able to operate equipment and understand the functions to facilitate the different clinical exams after a limited formal or informal initial learning period. Initial learning is supported by follow-up practice.
Medical team coordination	Caregivers’ relationship with each other and team presence	The extent to which the consulting doctor and other medical staff involved in the encounter work together like a team in conducting the telemedicine exam.
Clear future directives	Follow-up and end of visit instructions	The extent to which the consulting doctor instructs the patient regarding what to do after the encounter is over. Clear future directives include such things as medical instructions, who to contact in the future, will there be another telemedicine exam, will there be an in-person exam, and a status summary at the end of the encounter.
Conveys access/review of medical records	Records – practitioners agreed it was really the assurance that providers had reviewed records rather than just physically having the records	The extent to which the consulting doctor indicates to the patient that he/she has reviewed the patient’s records and has access to the records during the encounter.
Professionalism (clinician in patient room)	Qualifications	The extent to which the medical person in the room with the patient (eg, nurse practitioner) is professional and courteous toward the patient.
Mix with in-person exams	First visit in person	The degree to which relationship building and other data gathering during in-person meetings is needed to provide a necessary foundation for telemedicine encounter communications. The medical providers should assess the need for in-person meetings. The need may range from no meetings in person, only the first meeting in person, to intermittent in-person meetings.