Telehealth versus face-to-face visits: A comprehensive outpatient perspective-based cohort study of patients with kidney disease.

BACKGROUND: Telenephrology has become an important health care delivery modality during the COVID-19 pandemic. However, little is known about patient perspectives on the quality of care provided via telenephrology compared to face-to-face visits. We aimed to use objective data to study patients' perspectives on outpatient nephrology care received via telenephrology (phone and video) versus face-to-face visits.
METHODS: We retrospectively studied adults who received care in the outpatient Nephrology & Hypertension division at Mayo Clinic, Rochester, from March to July 2020. We used a standardized survey methodology to evaluate patient satisfaction. The primary outcome was the percent of patients who responded with a score of good (4) or very good (5) on a 5-point Likert scale on survey questions that asked their perspectives on access to their nephrologist, relationship with care provider, their opinions on the telenephrology technology, and their overall assessment of the care received. Wilcoxon rank sum tests and chi-square tests were used as appropriate to compare telenephrology versus face-to-face visits.
RESULTS: 3,486 of the patient encounters were face-to-face, 808 phone and 317 video visits. 443 patients responded to satisfaction surveys, and 21% of these had telenephrology encounters. Established patients made up 79.6% of telenephrology visits and 60.9% of face-to-face visits. There was no significant difference in patient perceived access to health care, satisfaction with their care provider, or overall quality of care between patients cared for via telenephrology versus face-to-face. Patient satisfaction was also equally high.
CONCLUSIONS: Patient satisfaction was equally high amongst those patients seen face-to-face or via telenephrology.

Introduction

In early 2020, the novel Coronavirus (COVID-19) emergency resulted in the cancellation of more than 70% of face-to-face patient visits [1] as people sought to avoid infection by staying away from healthcare institutions, and many states instituted stay at home orders. Patients and providers had to find ways to connect while avoiding exposure to the virus. At the same time, the Centers for Medicare & Medicaid Services (CMS) temporarily broadened access to telehealth services, allowing all Medicare patients to request telehealth visits and providers to be reimbursed for more telehealth services [2]. As a result, in March–April 2020, consumer and provider adoption of telehealth services as a replacement for cancelled face-to-face visits increased 300-fold at the height of the pandemic when compared to 2019 [3].

Telehealth has become a valuable tool to leverage specialized medical care not just in areas with limited access to expert providers such as nephrologists, neurologists and intensivists but also during extraordinary circumstances like the COVID-19 pandemic which demands social distancing measures. There is, however, a dearth of objective data on patient perceptions of the quality of care provided via telehealth, and in particular, telenephrology, compared to patient perceptions regarding quality of care provided via face-to-face visits. The types of medical services and patients that would be appropriate for telenephrology remain unclear. There is limited data on the types of telenephrology health care delivery models (telehealth only versus face-to-face interactions supported by telehealth) that would best serve the needs of patients while reducing health care costs. There is also still little known regarding long term patient and provider perspectives on the use of telenephrology.

In this retrospective study, we review the experience of the Division of Nephrology & Hypertension at Mayo Clinic, MN, with the use of telenephrology for outpatient visits from March to July 2020. We compared patient satisfaction across telenephrology (phone or video) and face-to-face visits utilizing a standardized structured uniform survey methodology. We evaluated the patient characteristics, spectrum of kidney diseases with diagnosis codes and geographical locations of patients served via telenephrology.

Methods

Clinic model and setting

Mayo Clinic is a multispecialty academic primary care and quaternary referral center, with comprehensive kidney care provided for the entire local population as well as serving as an international destination medical practice across multiple specialties. The Mayo Clinic Rochester Division of Nephrology and Hypertension evaluates and manages acute consultative as well as longitudinal care of patients with all manner of kidney conditions.

This retrospective survey-based study was limited to patients seen in the outpatient clinic of the Division of Nephrology & Hypertension at Mayo Clinic, Rochester, MN. Kidney and/or pancreas transplant, inpatient and end-stage kidney disease patient visits were excluded from the study. The providers included 48 Nephrology physicians and 4 Nephrology nurse practitioners (NP); the NPs saw patients in a specialized chronic kidney disease (CKD) clinic that serves a local population.

The study was approved by the Mayo Clinic Institutional Review Board and deemed to be minimal risk research. However, only patients who were at least 18 years of age and had consented through the Minnesota Research Authorization process that their medical data be used for research were included in the study. Patient demographic information that included age, gender, race, and home geographical location were obtained from Epic EHR software (Epic Systems, Verona, WI). We used the results of the Press Ganey Medical Practice Survey obtained through the Mayo Clinic Office of Patient Experience. Each survey result was linked to a patient medical record number and date of Service. Nephrology billing data was obtained and, using medical record number and date of service, a unique ID that allowed us to identify the primary billing diagnosis from the revenue data was created. Individual patient responses to survey data were not shared with the investigation team to protect the confidentiality of the survey respondents, and insights are only shared at an aggregate level. Primary diagnoses were prospectively categorized into seven distinct groups, namely BP/Hypertension, CKD stage 1–3, CKD stage 4–5, cystic kidney disease, nephrolithiasis, kidney parenchymal disease, and other. See S2 Table in S1 File for comprehensive list of primary visit diagnoses. Patients could have multiple diagnoses and there was overlap in patients’ diagnoses but for the purposes of our research, the primary diagnoses noted in the billing code supersede other kidney diagnoses for that particular visit.

Patient satisfaction data

The Mayo Clinic Office of Patient Experience (OPE) Team uses Press Ganey (South Bend, IN), a third-party vendor, to create patient satisfaction surveys, randomly select patients to receive the surveys, and to collect and analyze patient satisfaction data. The surveys have been tested in 3,361 patients from five organizations over a five month period [4] and examined for validity, reliability and readability [5-8]. Surveys were emailed to patients seen either by phone, video or face-to-face. Each provider at Mayo Clinic, Rochester, MN is limited to having 30 of their patients surveyed per month though they may see more than 30 patients per month. Each patient receives only one survey for a visit to Mayo Clinic, irrespective of the number of medical specialties they visit. Patients had up to six weeks to respond to the surveys. The survey questions were grouped into 4 domains of patient experience, namely (1) access (2 questions), (2) care provider (5 questions), (3) telemedicine technology (3 questions), and (4) overall assessment (2 questions). S3 Table in S1 File lists the question content summaries and the patient experience domains assessed. Patients answered individual questions with a categorical 5-point Likert scale ranked from very poor (1), poor (2), fair (3), good (4) and very good (5). The Mayo Clinic OPE team provided aggregated patient survey reports for patients seen in the Rochester Division of Nephrology & Hypertension outpatient clinic from March 1st–July 31st 2020. The aggregated survey data presented the frequency of each of the 5-point Likert scale responses for each question for each patient who responded to the survey for a particular visit for outpatient Nephrology care.

Outcomes

The pre-determined primary outcome was the percentage of patients who gave a score of good (4) or very good (5) on a Likert scale; a score of good (4) or very good (5) represent the two highest levels of patient satisfaction for each survey question. The pre-determined secondary outcome was the percentage of patients who scored very good (5), representing the highest level of patient satisfaction possible.

Statistical analyses

Continuous variables are presented as median (IQR), and categorical variables are described as number (percent). Phone, video, telenephrology (phone and video), and face-to-face visits are detailed separately, but were analyzed only as telenephrology versus face-to-face. This combination of phone and video was validated by examining phone versus video responses and finding no significant differences. Wilcoxon rank sum and chi-square tests were used, as appropriate. Survey results were presented as number (%) that answered good (4) or very good (5) but were analyzed on an ordinal (1–5) scale using the Wilcoxon rank sum test. Analyses were done using SAS v9.4 (SAS Institute Inc., Cary, NC).

Results

Participant characteristics

There were 10,079 outpatient nephrology clinic encounters at Mayo Clinic, Rochester MN between March 1st and July 31st, 2020. After applying the exclusion criteria, 4,611 patient encounters remained to be considered for the retrospective study (Fig 1). In 2020, 54.8% of the patients seen in the outpatient setting used Medicare/ Medicaid insurance.

Fig 1

Patient encounter selection criteria.

This schematic outlines the patient selection criteria.

Out of those included in the study, 3,486 (75.6%) of the patient were conducted face-to-face, 317 (6.9%) via video and 808 (17.5%) by phone. Established patients were defined as patients who had had at least one prior visit within the last three years with the nephrology department. Established patients comprised 941 (83.6%) of the telenephrology visits compared to 2104 (60.4%) of the face-to-face visits. There was a statistically significant difference between the telenephrology and face-to-face groups with respect to the type of patient encounter and age; the telenephrology patients were more likely to be established patients (p-value <0.001) and younger (median 65.5 years (IQR 51.8, 75.3) versus 66.7 (IQR 53.8, 75.3) (Table 1). There was no significant difference in gender or race between the telenephrology and face-to-face cohorts. The majority of the patients in the study were white.

Table 1

Study cohort patients’ characteristics (March, 2020 –July, 2020).

Characteristics	Telenephrology			F2F	Total visits	P-Value
	Phone visits	Video visits	Combined
Number of visits, n (%)	808 (17.5%)	317 (6.9%)	1125 (24.4%)	3486 (75.6%)	4611
n (%) of established patients	737 (91.2%)	204 (64.4%)	941 (83.6%)	2104 (60.4%)	3045	<0.001
Age, Median (IQR), years	66.2	62.7	65.5	66.7		0.015
	(53.5, 75.0)	(47.4, 72.1)	(51.8, 73.7)	(53.8, 75.3)
Gender n (%)						0.880
Female	391 (17.8%)	146 (6.7%)	537 (24.5%)	1655 (75.5%)	2192
Male	417 (17.2%)	171 (7.1%)	588 (24.3%)	1831 (75.7%)	2419
Race, n (%)						0.510
White	740 (17.5%)	294 (7.0%)	1034 (24.5%)	3182 (75.5%)	4216
Other	68 (17.2%)	23 (5.8%)	91 (23.0%)	304 (77.0%)	395

*p-values reflect Telehealth vs F2F groups, and are from Wilcoxon rank sum or chi-square tests as appropriate. F2F = face to face. Telenephrology (Combined) includes both phone and video visits.

Survey results

Out of the 4,611 encounters, there were 3455 unique patients in the study cohort. 443 patients responded to the Mayo Clinic Office of Patient Experience surveys. The survey response rate for division for the period October 2019 –September 2020 was 30.2% (S5 Table in S1 File). 350 (79%) of the 443 patients were seen via face-to-face visits compared to 93 (21.0%) via telenephrology visits. Like in the general study cohort, a higher proportion (79.6%) of telenephrology visits were by established patients, while only 60.9% of the face-to-face patients who responded to surveys were established patients. A high majority of the patients are white and there was a significant difference in race between the telenephrology and face-to-face patients who responded to surveys (p-value 0.007) (Table 2).

Table 2

Surveyed patients’ characteristics (March–July 2020).

Characteristics	Survey respondents
	Telenephrology			F2F	P-value
	Phone visits	Video visits	Combined
Number of visits	55 (12.4%)	38 (8.6%)	93 (21.0%)	350 (79.0%)
n = 443
n (%) of established patients	48 (87.3%)	26 (68.4%)	74 (79.6%)	213 (60.9%)	<0.001
Age, Median (IQR), years	70.6	67.3	69.1	69.7	0.397
	(61.2, 77.6)	(56.2, 72.1)	(58.1, 75.7)	(61.3, 77.1)
Gender, n (%)					0.100
Female	30 (16.1%)	16 (8.6%)	46 (24.7%)	140 (75.3%)
Male	25 (9.7%)	22(8.6%)	47 (18.3%)	210 (81.7%)
Race, n (%)					0.007
White	51 (12.0%)	34 (8.0%)	85 (20.0%)	341 (80.0%)
Other	4 (23.5%)	4 (23.5%)	8 (47.0%)	9 (53.0%)

*p-values reflect Telehealth vs F2F groups, and are from Wilcoxon rank sum or chi-square tests as appropriate. F2F = face to face. Telenephrology (Combined) includes both phone and video visits.

Using a five-point Likert scale ranging from 1 = very bad to 5 = very good, surveyed patients expressed their perspectives with regard to access to their nephrologist, how they related to their care provider, and when relevant–their opinions on the telemedicine technology–and their overall assessment of the care they received during the nephrology visit.

There was no significant difference in patient survey responses for the above when telehealth was compared to face-to-face visits. A large majority of patients selected a Likert score of at least 4 or 5 for all survey questions (Table 3).

Table 3

Frequency of survey top box scores: Likert score of 4 (good) or 5 (very good).

		Frequency of Top Box = 4 or 5 responses
		Telenephrology			F2F
		Phone	Video	Combined
		n (%)	n (%)	n (%)	n (%)	P-value
ACCESS	Ease of scheduling your appointment	49	38	87	327	0.056
		(90.7%)	(100%)	(94.6%)	(95.1%)
	Ease of contacting us (e.g. email, phone, web portal)	53	37	90	324	0.523
		(98.1%)	(97.4%)	(97.8%)	(94.7%)
CARE PROVIDER	Concern the care provider showed for your questions or worries	53	36	89	342	0.692
		(98.1%)	(94.7%)	(96.7%)	(98.6%)
	Explanations the care provider gave you about your problem or condition	54	35	89	341	0.637
		(100.0%)	(92.1%)	(96.7%)	(98.0%)
	Care provider’s efforts to include you in decisions about your care	53	34	87	341	0.839
		(98.1%)	(89.5%)	(94.6%)	(98.3%)
	Care provider’s discussion of any proposed treatment (options, risks, benefits, etc.)	53	34	87	342	0.400
		(100.0%)	(89.5%)	(95.6%)	(99.1%)
	Likelihood of your recommending this care provider to others	53	35	88	339	0.977
		(98.1%)	(92.1%)	(95.7%)	(96.9%)
TELEMEDICINE TECHNOLOGY	Ease of talking with the care provider over the video or audio connection	52	36	88
		(96.3%)	(94.7%)	(95.7%)
	How well the audio connection worked during your visit, whether by phone or video	53	36	89
		(98.1%)	(94.7%)	(96.7%)
	If you had a video visit, how well the video connection worked		37	37
			(97.4%)	(97.4%)
OVERALL ASSESSMENT	How well the staff worked together to care for you	53	36	89	337	0.998
		(98.1%)	(94.7%)	(96.7%)	(98.3%)
	Likelihood of your recommending our practice to others	54	37	91	340	0.674
		(100.0%)	(97.4%)	(98.9%)	(98.3%)

*p-values reflect continuous scales and are compared between telehealth and F2F visits. A Wilcoxon Rank Sum test was used. F2F = face to face; Telenephrology (Combined) = Phone + Video

At the deployment of the video visits modality, patient satisfaction with the telenephrology technology was closer to 80%, suffered an apparent decline after the initial month but rebounded within three months (Fig 2, Table 4). Of the 4611 outpatient nephrology encounters completed from March, 2020 –August 2020, there were 3455 patients, with 823 having more than one visit during the study period. 299 of the 823 were novel patients returning within the same initial encounter for short follow up visits.

Fig 2

Graph showing the frequency of trop box score of 4 or 5 by month for telenephrology responses.

Graphical illustration of satisfaction of audio, video and overall ease of communication with providers by survey month.

Table 4

How the frequency of top box score of 4 or 5 varied by month for telenephrology survey responses.

Month	Ease of talking with the care provider over the video or audio connection			How well the audio connection worked during your visit, whether by phone or video			If you had a video visit, how well the video connection worked			Phone	Video
	Top box = 4 or 5 count	Total	%	Top box = 4 or 5 count	Total	%	Top box = 4 or 5 count	Total	%	n = 808	n = 317
March	10	10	100%	10	10	100%				124	2
April	28	32	88%	31	32	97%	9	9	100%	331	62
May	15	15	100%	14	15	93%	7	7	100%	112	73
June	22	23	96%	21	23	91%	10	11	91%	141	92
July	13	13	100%	13	13	100%	11	11	100%	100	88

Travel distance saved by patients

19% (n = 214) of the patients seen via telehealth were from Rochester, MN while a majority were from outside Rochester but within 500 miles (n = 757, 67%). 13% (n = 148) were from more than 500 miles from Rochester, MN but within US boundaries, and only 1% (n = 6) were international patients. Fig 3

Fig 3

Number and percent of telenephrology visits by geography.

The figure graphically represents the plurality of locations served by telenephrology.

Distribution of primary diagnoses

The patients were grouped into seven pre-determined primary diagnoses based on billing information. The primary diagnoses were CKD stage 1–3, CKD stage 4–5, BP/ Hypertension, kidney parenchymal disease, nephrolithiasis, cystic kidney disease, and other. CKD accounted for 36.5% (n = 411, 214 for CKD stage 1–3 and 197 for CKD stage 4–5) of the patients seen via telehealth, and the majority were seen via phone visits (Fig 4). Among the survey respondents, CKD accounted for 37% of patients seen via telehealth (n = 35, 24 for CKD stage 1–3 and 11 for CKD stage 4–5) (S2 Fig in S1 File).

Fig 4

Patient distribution by visit type and diagnoses groups (all patients, face to face and telenephrology).

The majority of patients in the study were best characterized as CKD, however, the entire spectrum of clinical visits was supported by telenephrology.

Discussion

This novel study with objective patient-based findings highlights an initial attempt to focus on the value equation of telenephrology. These authors are unaware of any published study set with specific patient satisfaction metrics in a comparison of telenephrology versus face-to-face Nephrology care. This retrospective survey-based cohort study demonstrates no observable significant difference in patient perceived access to health care, satisfaction with their care provider, or overall quality of care between telenephrology and face-to-face visits. Patient overall satisfaction was also equally high in telenephrology as compared to face-to-face visits. We also did not see any significant difference in patient perspectives when we compared patients evaluated via phone versus video visits.

Of note, over 87% of the phone compared to 68% of the video visits in the surveyed cohort were with established patients; although there was no observed statistically significant difference, satisfaction among the phone patients was also generally higher compared with video visit patients. The reason for the higher satisfaction among the phone patients deserves further study. This may be due to already established patients having favorable opinions about this medical center, its providers and perhaps through previous video and phone experiences. This finding is consistent with multiple research articles in which patient-physician familiarity is found to be associated with higher patient satisfaction [9-11].

Familiarity with and the ease of use associated with technology affected patient satisfaction. Satisfaction with video connection was initially high when the video program was rolled out. However, this dropped the following two months and improved back to high levels following an institution-wide process flow change. This institutional process change involved the video call setup being switched from an institutional centralized system to each department managing their own video setup process.

Additionally, improved satisfaction with the video technology may suggest the occurrence of a learning curve for the providers, through the 317 video visits over the span of five months. We speculate that patients’ satisfaction with video visit technology also improved as they encountered more exposure to the virtual technology. We saw that 23% (n = 823) of our patients had more than 1 visit with the division during the 5-month period of the study. Hypothetically, patients may have become adept at using video technology, since they likely had video visits with other providers in other departments, institutions or even with family members, as tele-video visits became synonymous with social distancing.

There still remains a dearth of data on what type of diagnoses and types of patients are best served by telehealth. Our retrospective study shows CKD 1–5 made up 36.5% (n = 411), contributing the most to the number of telenephrology visits. 48% (n = 590) of the telehealth visits were also by patients who live over 100 miles from our medical center and 32% (n = 363) were from within 100 miles while only 19% (n = 214) were from Rochester MN. Our findings show that most of the telehealth patients live further away from Mayo Clinic and are generally more satisfied if they do not have to travel long distances to visit with their physicians.

This study has various limitations. Although overall satisfaction with telenephrology was high (>96%) and not statistically significantly different from face-to-face visits, the overall patient satisfaction survey response rate was quite low, introducing potential self-selection bias. However, these patient experience survey results are similar to the VA telehealth satisfaction results, where patient satisfaction among veterans was found to be equally high, with 92% for clinical video telehealth [12]. Since the Mayo Clinic patient experience team sends only one survey per patient and patients are more likely to have visits with more than one department, patients’ experiences with other divisions likely contributed to their perceptions of telehealth. Our study therefore includes patients who were attended to in the Nephrology & Hypertension division but may have been seen by other medical departments too. To minimize the effect of recall bias, survey participants had up to 6 weeks to respond to the surveys. The risk of misclassification bias exists since we grouped the phone and video visit patients together, although the majority of the telenephrology visits were phone visits and also most of the phone visits were with established patients. We did compare the patient responses for phone vs video visits and saw no statistically significant difference between the two groups. This may suggest that combining phone and video visits was appropriate, but still deserves further research in the future. We also did not have the characteristics of survey non-respondents to compare how they may differ from respondents.

Survey-studied patient satisfaction is but one of the major ways of interpreting the overall value equation of patient outcomes. This study was limited in its ability to determine cost of care metrics and overall effectiveness of the care delivered. This latter issue is difficult to measure, but applications of guideline-driven medication prescription and care outcomes such as accepted metrics of diabetes, blood pressure and glomerular filtration rate (GFR) decline deserve tracking once more robust datasets are accumulated.

Our study period was also marked by the COVID-19 pandemic which on one hand spurred the widespread increase in the use of telehealth as a modality of health care delivery, but it may also suggest that patients were more accepting of telehealth services at this time opposed to when there are no concerns associated with risks posed by a pandemic. Potentially, patients’ willingness to participate in telenephrology rather than face-to-face visits may drop once people no longer need to social distance. Whether acceptance of telenephrology for patients who live over 100 miles away and have limited access to nephrology care due to distance and travel costs will remain attractive deserves study as well in a post-pandemic era. The risk of acquiescence further complicates the results of our analyses. We saw many patients scoring good (4) or very good (5) for most questions and this may suggest the phenomenon whereby the patients agreed with all the questions or measures in the survey. Even though we used a survey that was designed and tested to reduce such bias, the risk and its inherent effect on interpreting the results persist.

The data presented highlight a difference in patient satisfaction over the time period of the study. The apparent decrease in time-based patient ratings with an initial high relative acceptance followed by a brief few weeks decline and ultimately a surge in high acceptance and satisfaction scores deserves further study and discussion. This was unlikely to have altered the overall outcomes based on the numbers of responses over these time frames. A few possible explanations may be entertained and may serve to highlight potentially promising avenues for future research efforts into various virtual practice processes. The initial time frame of telenephrology and other service lines at this institution were characterized by a central “rooming” process. This process was ultimately transitioned to the local service areas.

Its provisional and single center nature notwithstanding, the overall conclusion of our study is that patient satisfaction was equally high amongst those patients seen face-to-face or via telenephrology. Whether telenephrology may be better-suited for established patients who are already familiar with the health care organization and providers, or for patients with specific diagnoses–and how the overall outcomes of clinical performance compare–will deserve robust study.

This study provides objective positive evidence that fostering telenephrology visits adds arguably the most crucial initial step necessary for more complete implementation of virtual care models in Nephrology.

S1 Table: Number of available patient surveys per diagnosis group. Distribution of diagnostic categories and survey responses; S2 Table: Diagnoses Grouping of patients seen in Outpatient nephrology March–July 2020. Overall diagnoses by number of patients seen in the division of Nephrology March through July 2020; S3 Table: Outpatient Nephrology Survey. Patient- facing survey tool; S4 Table: How the frequency of top box score of 5 varied by month for telenephrology survey responses. Variability of top box score: 5 only, reported by month; S1 Fig: Press Ganey Survey—Medical Practice Survey Response Rate for October 2019 –September 2020, Nephrology Division, Mayo Clinic, Rochester MN. Graphical form of the variability of the top box scores (4 or 5) by month; S5 Table: Press Ganey Survey—Medical Practice Survey Response Rate for October 2019 –September 2020, Nephrology Division, Mayo Clinic, Rochester MN. Nephrology provider numbers in the division of Nephrology and the survey response rates; S2 Fig: Survey respondents distribution by visit type and diagnoses groups (Face-to-face and telenephrology). Respondents by diagnostic groups; S6 Table: Survey Respondents by primary diagnosis group. Distribution of respondents by face to face and telemedicine.

(PDF)

Click here for additional data file.

27 Oct 2021

Please submit your revised manuscript by Dec 11 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

17 Feb 2022

Response to Reviewers

Please see below for responses to the reviewers’ comments/ questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: With the current pandemic the need for more telemedicine to serve the population has become more obvious. There are quite a few biases in the results but the authors do a good job of acknowledging them in their discussion which includes small survey size, mostly established patients.

Also given that the authors don't know if the survey pertained to specifically the nephrology visit would it not be prudent to change the title to NOT include nephrology?

Response: All the patients included in our study have kidney disease and visited with our Nephrology & Hypertension division at least once during the study period. These patients commonly have multiple comorbidities requiring care by a multidisciplinary team spanning various medical specialities. As such, we have changed the title of the paper to be “Telehealth versus Face-to-face visits: A Comprehensive Outpatient Patient Perspective-based Cohort Study of Patients with Kidney Disease”

Also most telehealth visit patients lived further away and are generally known to be more satisfied if they don't have to travel far to see their physician, I think that is worth a mention.

Response: This is true. We have now included it in the manuscript (fifth paragraph of discussion section).

And I like that they acknowledged that they couldn't measure patient outcomes which would be good to look at in the future and see if they correlated with the patient satisfaction scores

Response: Understanding patient outcomes, patient and provider perspectives on the different models of telehealth will be very important in figuring out how to improve care delivery to patients. It will be useful to study these factors not just within our institution but nationally and globally too.

Reviewer #2: This is a retrospective single center study using a standardized structured survey to assess patient satisfaction, comparing telenephrology versus face-to-face visits in outpatient nephrology and hypertension clinic. Patient satisfaction was equally high for the telenephrology and face-to-face visits. Telenephrology was more likely to be done for established patients and younger patients. The majority of patients receiving telenephrology were found to reside further from clinic.

These findings were not surprising, but this study confirmed what we experience. There were 52 providers in this study which provided good sampling of the variability among providers. The authors carefully designed the study to assess the survey. The authors also acknowledged the limitations of the study and discussed the future studies that are needed.

Major points

Who decided the modality for each visit, the provider or the patient?

Response: This was mostly based on patient preference. Providers encouraged initial consults to be face-to-face or video. Established patients could choose face-to-face, video or telephone, based on their preference and medical needs. Providers could triage the cases prior to the appointments being set and provide guidance on the appropriateness of the modality, after patients make their preferences known.

Why did the authors reduce a 5-point Likert scale to a binary scale (i.e., good (4) or very good (5) vs. 1-3)?

Response: Our institution uses Likert scale survey questionnaires to evaluate patient experiences. The results are then summarized using top-box score analyses that combines the top two responses to create a single number. The institution focuses on service, quality, and excellence and holds providers accountable to achieving top box score experience when it comes to patient satisfaction.

It may worthwhile to analyze the repeat visits.

Response: At the moment, we do not have enough data, given the study period, to effectively analyze repeat visits. The COVID-19 pandemic also continues to rage on, and this may affect patients’ perspectives on telehealth. However, as telehealth persists and patients continue to seek care at our institution, we will have the longitudinal data and we are excited to share our findings over time.

Could the authors further discuss the changes in telenephrology from pre-COVID to COVID times, in addition to the race differences mentioned? Are there any relevant studies that should be cited in this regard?

Response: Prior to the COVID-19 pandemic, CMS payment requirements limited telehealth reimbursement. As such, many patients and providers did not have access to telehealth. However, COVID-19 pandemic led to the presidential order that directed CMS to expand telehealth services under the 1135 waiver authority and Coronavirus Preparedness and Response Supplemental Appropriations Act. The opportunities to effectively study racial differences in the use of telehealth given access barriers, literacy/language issues along with acceptance is a goal this research team hopes to address in upcoming work.

Minor

Table 2 legend should be October 2019 - September, 2020.

Response: The study period was from March, 2020 – July, 2020

________________________________________

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous, but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

23 Feb 2022

Telehealth versus Face-to-face visits: A Comprehensive Outpatient Perspective-based Cohort Study of Patients with Kidney Disease

PONE-D-21-23986R1

Dear Dr. Albright,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Abhishek Makkar, M.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

1 Mar 2022

PONE-D-21-23986R1

Telehealth versus Face-to-face visits: A Comprehensive Outpatient Perspective-based Cohort Study of Patients with Kidney Disease

Dear Dr. Albright Jr:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Abhishek Makkar

Academic Editor

PLOS ONE