Characterizing the impact of health literacy, computer ability, patient demographics, and portal usage on patient satisfaction with a patient portal.

OBJECTIVE: We sought to measure patient portal satisfaction with patient portals and characterize its relationship to attitude towards computers, health literacy, portal usage, and patient demographics.
MATERIALS AND METHODS: We invited 13 040 patients from an academic medical center to complete a survey measuring satisfaction, perceived control over computers, and health literacy using validated instruments (End User Computing Satisfaction, Computer Attitude Measure, and Brief Health Literacy Screen). We extracted portal usage and demographic information from the medical center data warehouse.
RESULTS: A total of 6026 (46.2%) patients completed the survey. The median (IQR) scores for satisfaction, computer control, and health literacy were 87% (20%), 86% (22%), and 95% (15%), respectively. The normalized mean (SD) usage of messaging, lab, appointment, medication, and immunization functions were 6.6 (2.6), 4.6 (2.4), 3.1 (1.7), 1.5 (1.2), and 0.88 (0.91) times, respectively. Logistic regression yielded significant odds ratios [99% CI] for computer control (3.6 [2.5-5.2]), health literacy (12 [6.9-23]), and immunization function usage (0.84 [0.73-0.96]). DISCUSSION: Respondents were highly satisfied and had high degrees of computer control and health literacy. Statistical analysis revealed that higher computer control and health literacy predicted higher satisfaction, whereas usage of the immunization function predicted lower satisfaction. Overall, the analytical model had low predictive capability, suggesting that we failed to capture the main drivers of satisfaction, or there was inadequate variation in satisfaction to delineate its contributing factors.
CONCLUSION: This study provides insight into patient satisfaction with and usage of a patient portal. These data can guide the development of the patient portal, with the ultimate goal of increasing functionality and usability to enhance the patient experience.

BACKGROUND AND SIGNIFICANCE

Patient portals are electronic personal health records that are typically tethered to an electronic health record (EHR) system. They allow patients and their proxies to access health information from the EHR, and they may also provide users the ability to view and send secure communications, request appointments, and pay medical bills to the healthcare system. Their development and adoption have been driven, in part, by the Health Insurance Portability and Accountability Act (HIPAA) and the Center for Medicare and Medicaid Services (CMS) Health Information Technology for Economic and Clinical Health (HITECH) Act’s “Meaningful Use” objectives. In 2013, a private market research and consulting organization estimated that 50% of US hospitals and 40% of US physicians had a secure patient portal implemented,, and this number is forecasted to increase to more than 75% by 2020.,

Many studies have characterized the effect of patient portals on disease management and primary prevention., Earlier studies possessed a wide variety of study designs, portal functionalities, and implementation processes that may have obscured the effect patient portals had on health outcomes. More recent evidence is accumulating from rigorous studies with historical cohorts and randomized controlled trials on the effectiveness of patient web portals for improving health outcomes, such as in the management of patients with diabetes, hypertension, and depression as well as in the use of preventative services such as cancer screenings, vaccinations, cardiovascular disease screening, sexually transmitted infections, and weight loss counseling.,

The body of research to date suggests that patient portals can support quality improvement and healthcare cost containment strategies., However, any utility provided by patient portals can only be leveraged if the system is adopted and its use is sustained., Well-designed patient portals can empower patients to manage their own healthcare by enabling them to better understand their health and strengthening communication between patient and provider. Conversely, poorly designed and managed patient portals can discourage its use, negatively impact provider productivity, or even compromise patient safety. For this reason, it is important to characterize the perceptions of different stakeholders in the implementation and continued development of patient portal.,

Currently, there remains a lack of research characterizing patient satisfaction with the patient portal. There are several theoretical frameworks that attempt to characterize end-user reactions to health information technology. The Technology Acceptance Model (TAM) has been used extensively to predict and explain end-user reactions of providers to health information technology, including the use of patient portals. The TAM hypothesizes that intent to use and actual use of a technology is based on the individuals’ perceived ease of use and perceived usefulness of that technology. On the other hand, end-user satisfaction serves as a perceptual measure of system success; it posits that an information system that meets the needs of its users reinforces users’ satisfaction with that system. In other words, satisfaction with the patient portal predicts increased use of the patient portal, and in this context, enables the benefits outlined above.

Furthermore, although health literacy and computer ability have been identified as critical components of portal adoption, there are few studies that examine their relationship to patient portal satisfaction., Health literacy is an essential competency needed to make informed health decisions, obtain and interpret health information, and improve quality of life. Computer literacy, on the other hand, is needed by users to effectively participate in contemporary society due to the continuing expansion of electronic information and communication like patient portals., Prior work has also identified disparities in the social factors predicting the use of patient portals, including demographics and access to broadband internet., In order to better understand the patient experience, it is important to measure patient portal satisfaction in the context of patients’ health literacy, computer ability, and demographics.

OBJECTIVE

In this study, we sought to characterize patient satisfaction with and usage of the patient portal at a large academic primary and tertiary medical center located in Nashville, TN. Combining elements of the TAM and end-user satisfaction, our objectives are as follows:

Measure patient portal satisfaction using a validated survey tool,

Measure patient portal use directly, and

Add context to this information by including patients’ demographics, health literacy, and computer ability

We sought to determine how these factors affected patient portal satisfaction (Figure 1). We hypothesized that health literacy, computer ability, and usage would correlate with satisfaction.

Figure 1.

Theoretical framework of measured characteristics. Patient (end-user) satisfaction with the patient portal was measured, along with health literacy, computer control, demographics, and actual usage. Perceived ease of use was measured as a subscale of the patient satisfaction score. The dashed lines indicate aspects that align with the TAM. TAM, Technology Acceptance Model.

MATERIALS AND METHODS

Study setting

The study was conducted at Vanderbilt University Medical Center (VUMC), a large academic primary and tertiary care facility with local and regional referral bases. VUMC cares for over 600 000 unique patients, conducts over 40 000 surgical operations, 1.6 million ambulatory visits, and 70 000 Emergency Department visits annually. During this study period, VUMC offered patients and their proxies an institutionally developed patient portal called My Health at Vanderbilt (MHAV), which was initially launched in 2003 and deployed widely throughout the institution in 2007.,, This patient portal allowed patients to send and receive secure messages to and from their providers, request to schedule appointments, view and pay their VUMC medical bills, and view EHR data such as medications list, allergies, immunizations, and certain laboratory results. MHAV and its connected EHR were certified for CMS Meaningful Use Stage 2. The patient portal in this study was developed in-house by the institution, but its feature set is shared among many of the most popular patient portal vendors including Epic Systems Corporation’s MyChart® and Cerner Corporation’s HealtheLifeSM., A more thorough description of MHAV’s feature set is described elsewhere.

We defined a study period from October 31, 2016 to November 1, 2017 to support comparative analyses between respondents and non-respondents. On November 2, 2017, the patient portal underwent a major overhaul as VUMC transitioned its EHR and tethered portal to a commercial vendor, and the scope of this study does not include patients’ experiences with this update. The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was approved by the Vanderbilt University Institutional Review Board (IRB).

Study population

As of November 1, 2017, a total of 402 777 users had signed up for MHAV. We recruited a pool of 13 040 patients who were part of a larger research cohort called My Research at Vanderbilt (MRAV). MRAV is a recruitment tool that reaches over 20 000 adult users of MHAV who have previously opted into receiving research requests from Vanderbilt researchers. To be included in our study, participants were required to have an email address listed with MRAV and completed at least one scheduled outpatient appointment at VUMC during our study period. From these parameters, we obtained a list of 13 040 patients that fit these criteria. On February 5, 2018, we emailed these patients an IRB-approved message that invited them to complete a 23-question survey hosted on Research Electronic Data Capture (REDCap) and collected responses for 13 weeks. Survey items from 3 previously validated questionnaires (described below and listed in Supplementary Table S1) were used to assess user satisfaction, computer control, and health literacy. For respondents who completed the survey, we extracted MHAV usage logs and personal demographic data from the Research Derivative, a VUMC electronic data warehouse dedicated to research activities. We defined usage as the act of accessing the feature on the patient portal through clicking on the webpage hyperlink for that feature, and we defined active users as those who had logged into their account and used at least one MHAV feature during the study period. Usage counts only include actions that are related to that users’ medical record; thus, actions such as a parent checking their child’s immunization records or a delegate granted access to their spouse’s account managing their messages are logged separately.

Survey instruments

The survey consisted of 3 validated instruments: the End User Computing Satisfaction (EUCS) survey, the Computer Attitude Measure (CAM), and the Brief Health Literacy Screen (BHLS). The EUCS survey tool is a standardized measure of satisfaction with a specific application and was chosen for its ease of use, brevity, and applicability to both research and practice. The tool was previously validated in the evaluation of end-user satisfaction with a web-based system. It is comprised of 12 items scored on a 5-point Likert scale measuring 5 factors: content, accuracy, format, ease of use, and timeliness. Possible scores range from 12 to 60. The tool’s internal consistency is 0.92, which reflects how strongly the items in the survey tool are measuring a single characteristic.

One definition of computer literacy is the ability to use computers confidently for obtaining needed information, solving specific problems, and performing data-processing tasks. This includes a fundamental understanding of the operation of computers in general as well as the use of several types of application software packages. There is a strong correlation between total computer ability and perceived control. In this study, we take the perceived control subscale of the CAM as a surrogate measure for computer ability. The subscale of this tool is made up of 7 items scored on a 7-point Likert scale. Possible scores range from 7 to 49, with an internal consistency of 0.89.

Health literacy is an important parameter that is linked not only to health service utilization and patient experience but also to socioeconomic status and health outcomes. The BHLS is a survey tool that has strong correlation with the Short Test of Functional Health Literacy in Adults (S-TOFHLA) tool and performs well for discriminating among adequate, marginal, or inadequate health literacy. This tool consists of 4 items scored on a 5-point Likert scale. Possible scores range from 4 to 20, with an internal consistency of 0.77.

Statistical analysis

We used Stata 15 for all data analysis and visualization. To determine how representative our sample was from the larger population from which they were drawn, we compared age, sex, race, and ethnicity of survey respondents, the MRAV cohort, active MHAV users, and all VUMC patients. The Kruskal-Wallis rank test was used to compare age due to the non-normal distribution and Chi-squared tests were used to compare sex, race, and ethnicity. Pairwise comparisons among these groups were made using one-way analysis of a variance, and a Bonferroni correction was used to correct for multiple comparisons. Survey scores were calculated as percentages of maximum scores, with higher scores indicating a greater degree of satisfaction, computer control, and health literacy.

Due to the high density of data collected, we used violin plots and density distribution sunflower plots to visualize data. Violin plots were used to summarize survey scores and usage data. Violin plots superimpose the estimated kernel density—analogous to a smoothed, continuous histogram—over the summary statistics displayed by a traditional box plot., Density distribution sunflower plots were used to visualize the bivariate relationships of satisfaction scores against computer control, health literacy, age, usage, and demographics. Sunflower plots are a type of heatmap that allows for the visualization of high-density bivariate data using additional visual elements that encode density. Usage data were transformed using a cube root function to reduce skewness for statistical analysis and ease visualization in the violin plots and sunflower plots.,

We performed ordered logistic regression with satisfaction as the outcome. Computer control, health literacy, demographics (age, sex, race, ethnicity), and usage of the messaging function, appointment function, laboratory function, immunization function, and medication function were covariates in this model. Usage counts were transformed with a cube root function and scaled by the interquartile range of this transformed data in order to reflect meaningful change in usage in the ordered logistic analysis. Ordinary least squares regression was attempted but the underlying assumptions were not met: although residuals were normalized by cube root transformation, residual variance remained heteroscedastic. Moreover, polynomial regression and restricted cubic spline regression did not improve upon the linear model. Thus, our inferential statistical analysis was limited to ordered logistic regression. We accept a type I error rate of 0.001 to provide a high threshold for concluding significance when it does not exist.

RESULTS

Survey response and demographics

A total of 6026 (46.2%) participants responded to the survey during the collection period (Figure 2). Among respondents, 5880 (97.6%) had used MHAV at least once during the study period. The differences in the demographic characteristics among survey respondents, the MRAV cohort, all MHAV users, and the general VUMC population are summarized in Table 1. In general, survey respondents were more likely to be older, female, and non-Hispanic white compared with each of the other groups. A similar trend is found when comparing each subsequent group to the larger encompassing groups (ie, patients in the MRAV cohort compared with all MHAV users and VUMC patients).

Figure 2.

Survey collection and respondent summary.

Table 1.

Comparison of survey respondent demographics with the larger population

	Respondentsa	MRAVb	MHAVc	VUMCd
	N=5880	N=21 287	N=179 010	N=645 909
Age (years)	61 (50–69)	56 (42–66)	48 (33–62)	39 (16–60)	H = 18f,	P < 0.001
Sex
Female	3640 (62%)e	13 692 (65%)	111 772 (62%)e	362 740 (56%)	Χ 2 = 207,	P < 0.001
Male	2240 (38%)e	7519 (35%)	67 238 (38%)e	283 169 (44%)
Race
Asian	74 (1.3%)	362 (1.7%)	4237 (2.4%)	11 064 (1.7%)	Χ 2 = 2.9f,	P < 0.001
Black	283 (4.8%)	1299 (6.1%)	13 280 (7.4%)	70 798 (11.0%)
Indigenous	6 (0.1%)	24 (0.1%)	308 (0.2%)	1157 (0.2%)
Unknown	95 (1.6%)	1307 (6.2%)	11 093 (6.2%)	130 783 (20.2%)
White	5422 (92.2%)	18 219 (85.9%)	150 092 (83.8%)	432 107 (66.9%)
Ethnicity
Hispanic	78 (1.3%)	339 (1.6%)	3564 (2.0%)	26 758 (4.1%)	Χ 2 = 51f,	P < 0.001
Not Hispanic	5677 (96.5%)	19 967 (94.1%)	158 917 (88.8%)	478 947 (74.2%)
Unknown	125 (2.1%)	905 (4.3%)	16 529 (9.2%)	140 204 (21.7%)
Grouped demographics
Non-Hispanic White	5285 (89.9%)	17 989 (84.8%)	140 158 (78.3%)	391 309 (60.6%)	Χ 2 = 19f,	P < 0.001
Non-Hispanic Black	279 (4.7%)	1280 (6.0%)	12 494 (7.0%)	64 748 (10.0%)
Other Non-Hispanic	238 (4.0%)	1603 (7.6%)	22 794 (12.7%)	163 094 (25.3%)
Other Hispanic	78 (1.3%)	339 (1.6%)	3564 (2.0%)	26 758 (4.1%)

Study participants who completed the survey and used the patient portal during the study period.

All patients who are part of MRAV, a recruitment tool that reaches over 20 000 adult users of MHAV who have previously opted into receiving research requests from Vanderbilt researchers.

All MHAV users who were active during the study period.

All patients served by VUMC during the study period.

Pairwise comparisons were insignificant.

Values are 103.

Abbreviations: MHAV, My Health at Vanderbilt; MRAV, My Research at Vanderbilt; VUMC, Vanderbilt University Medical Center.

Survey and usage data

Patient portal satisfaction had a left-skew and a bimodal distribution around 80% and 100%, with a median (interquartile range, IQR) score of 87% (20%). The median (IQR) scores for the content, format, ease of use, and timeliness satisfaction subscales were 80% (20%), and the accuracy subscale of the satisfaction score was 100% (20%). These subscales are visualized in Supplementary Figure S1. Computer control had a left-skewed and platylkurtic distribution, with median (IQR) scores of 86% (22%). Platylkurtosis describes the shape of a probability distribution and indicates fewer and less extreme outliers when compared with a normal distribution. Health literacy scores also had a left-skew and platylkurtosis, with median (IQR) scores of 95% (15%). These results are summarized in Figure 3A. Usage data for messaging, appointment, lab, immunization, and medication functions demonstrated a strong right-skew and platylkurtosis, with cube root transformation normalizing the distributions. The resulting frequency of the usage yielded a mean (standard deviation) of 6.6 (2.6) for the messaging function, 4.6 (2.4) for the labs function, 3.1 (1.7) for the appointment function, 1.5 (1.2) for the medication function, and 0.88 (0.9) for the immunization function. These results are summarized in Figure 3B.

Figure 3.

Summary of survey scores and usage. (A) Violin plots of patient portal satisfaction (EUCS), computer control (CAM), and health literacy (BHLS) scores are reported as percentages of maximum score, with median (IQR) scores of 87% (20%), 86% (22%), and 95% (15%). (B) Violin plots of usage data are scaled with a cube root function for ease of visualization. The mean (SD) usage of these functions during the study period was of 6.6 (2.6) for Messaging, 4.6 (2.4) for Lab Result, 3.1 (1.7) for Appointment, 1.5 (1.2) for Medication, and 0.88 (0.9) for Immunization

Statistical model

The bivariate relationship between satisfaction and the covariates studied depicts a nonlinear pattern (Figure 4). The ordered logistic regression model yielded computer control, health literacy, and usage of the immunization function as the only significant factors predicting patient portal satisfaction. The odds [99% confidence interval] of scoring satisfaction one point higher for each percent increase in computer control was 3.6 [2.5–5.2] (P < 0.001) with all other factors held equal. Similarly, the odds ratio was 12 [6.9–23] (P < 0.001) for health literacy and 0.84 [0.73–0.96] (P < 0.001) for usage of the immunization function. The model accounted for 1% of the overall variation in the sample data (pseudo-R2 = 0.01; Table 2). Generalization of this model yielded a nonlinear probability model predicting, at each level of satisfaction, the effect of computer control and health literacy on satisfaction. These results indicate that at lower levels of satisfaction, computer control had a much larger effect; at higher levels of satisfaction, health literacy had a larger role. These results are summarized in Supplementary Table S2.

Figure 4.

Relationship of satisfaction with patient characteristics and portal usage. Density distribution sunflower plots visualize the nonlinear, bivariate, relationships of satisfaction scores against computer control, health literacy, age, usage, and demographics. Darker regions (orange hexagons) with more petals (hash marks) indicate a higher density of responses.

Table 2.

Results of ordered logistic regression

	Odds ratioa	99% CI	P-valueb
Computer control	3.6	[2.6–5.2]	<0.001
Health literacy	12	[6.9–23]	<0.001
Usagec
Messaging function	1.02	[0.91–1.1]	0.69
Lab result function	1.05	[0.92–1.2]	0.35
Appointment function	1.06	[0.92–1.2]	0.29
Medication function	0.92	[0.78–1.1]	0.19
Immunization function	0.84	[0.73–0.96]	<0.001
Demographics
Age	1.0	[0.99–1.01]	0.33
Non-Hispanic White	1.2	[0.70–2.0]	0.43
Non-Hispanic Black	1.3	[0.73–2.3]	0.24
Other Non-Hispanic	0.74	[0.41–1.3]	0.18
Other Hispanicd	1	–	–
Sex
Female	1.03	[0.91–1.2]	0.52
Maled	1	–	–

OR [99% CI] denotes the odds of scoring satisfaction one level higher per one unit increase in the covariate, with all other factors held equal.

Overall model was significant (P < 0.001) with a pseudo-R2 of 0.01.

Usage counts are scaled and transformed by a normalizing factor.

Collinear factors omitted from the model.

DISCUSSION

In this study, we sought to measure patient satisfaction with and usage of the patient portal of a large academic primary and tertiary care center. The study evaluated how computer ability, health literacy, demographics, and usage correlated with patient portal satisfaction. This study incorporates aspects of the TAM and end-user satisfaction along with actual use measurements to characterize the patient experience. From our sample of 13 040, we observed an above average response rate of 46%. Survey respondents were highly satisfied with the patient portal and had a high degree of computer control and health literacy. Satisfaction with the accuracy of information in the portal, in particular, was very high (median satisfaction of 87%, IQR 78%–98%) for most users in our sample, which reflects the advantage of a tethered patient portal that imports data directly from the EHR (Supplementary Figure S1). The survey respondents most frequently used the Messaging, Appointment, and Labs functions during the study period. Logistic analysis revealed that users were more likely to score satisfaction higher with increasing levels of computer control and health literacy, whereas users who used the Immunization function more often were more likely to score satisfaction slightly lower.

Overall, the statistical model demonstrated low predictive capability. This suggests either that our measurement tools failed to capture the main drivers of satisfaction, or that there was inadequate variation in our sample’s characteristics to produce variation in satisfaction that would better predict the contributing factors to satisfaction. Nevertheless, these results can be interpreted in 2 main ways. One interpretation is that respondents believed that for MHAV was inherently satisfying to use, regardless of the user’s health literacy, computer control, demographics, or usage of the portal. This could indicate that the patient portal is accessible and functional for a wide variety of users. Conversely, another interpretation is that the study sample was highly selective, with respondents consisting primarily of users who actively used the patient portal, opted into MRAV, possessed and maintained an email account, and voluntarily completed the online survey. That is, respondents may not reflect the wider community of patient portal users. This may limit the generalizability of our findings to the more than 400 000 MHAV users and the larger population served by VUMC overall, whose demographics differ significantly from our sample. Indeed, in other studies, patient portal use has been demonstrated to be lower among minorities and publicly insured populations, suggesting that any benefits provided by patient portals also have the potential to widen disparities among the populations it serves.,, Even in our study’s select population, whose respondents have a high degree of computer ability and health literacy, those most dissatisfied also had the lowest scores in computer control and health literacy. Both the technology acceptance and end-user satisfaction models suggest that increasing the patient portal’s perceived usefulness and ease of use can lower these barriers.

This study has several limitations. Our data come from a single, large academic medical center with a locally developed patient portal, which may differ from other institutions’ portal implementations. However, MHAV has functionality similar to most other patient portals. As a cross-sectional study, we were unable to control for unmeasured, confounding variables. For example, the duration of engagement with the healthcare organization and its patient portal may influence satisfaction more than our measured variables. In using the research cohort, our study population is limited by membership bias. The limitations of self-reporting are also inherent to this study design, including a self-selection for those who have a strong opinion on the system and the unavoidable subjectivity in responses. And while the research cohort’s disease burden reflected that of the larger population that the institution serves, our study design did not examine patient-specific clinical data, such as healthcare utilization, insurance status, or diagnoses. Indeed, our sample’s demographics differ significantly from the overall population, which itself differs from other institutions. Finally, our data collection occurred several months after MHAV had received a substantive update, introducing recall bias. Ideally, the survey could have been released and responses collected prior to this patient portal update. It is possible that the survey respondents’ thoughts regarding the prior version of the patient portal were influenced by this update.

Despite these limitations, this study provides important information about the patient experience as well as additional insight into patient portal satisfaction. Just as a prior patient portal usage study in 2011 helped to direct the development of MHAV, this study of the adult ambulatory environment, along with recent similar studies of MHAV usage in the pediatric and specialty clinic setting,, can provide leaders of the health system guidance as they shape the policies and regulation that govern MHAV’s functionality and usage. Other institutions may use the results of this study by designing patient portals that support a broad spectrum of health literacy and computer ability. Further, these institutions may prioritize patient engagement efforts to improve and reinforce computer ability because these may be most relevant to patients that are the least satisfied. Our study suggests that health literacy and computer literacy heavily affect patient portal satisfaction, and these may serve as 2 metrics that should be considered in the design, implementation, and maintenance of patient portals. Because patient experience is essential to improving quality of healthcare,, patient portals must exhibit patient-centric design to promote sustained use.,, This study provides information that can help refine future studies about the patient experience as well as potential targets for systemic intervention.

Future directions for this project include a repeat measurement of satisfaction and usage with the dramatically overhauled and updated patient portal. The tools used in this study provide a robust method that allows for reliable, repeated measurements of satisfaction, computer ability, and health literacy. With 3072 (51%) of the survey respondents agreeing to be contacted again, we have the opportunity to repeat these measurements on the same users, enabling one-to-one comparisons of satisfaction and usage prior to this EHR migration and patient portal update. Further research could include focus groups of individuals with lower satisfaction scores to obtain a qualitative analysis of their experience. In addition, future work could more closely examine the relationship between satisfaction and users’ characteristics by contextualizing these to specific phenotypes to determine if certain disease processes are associated with higher or lower satisfaction. Furthermore, measurements of specific patient outcomes may reveal interesting relationships with health literacy, computer ability, or portal usage, and these findings could lead to interventions that aim to improve those outcomes. Lastly, further research into patient portal satisfaction could be refined with additional measurements to expand the ability to characterize patient portal satisfaction, such as factoring in the provider’s satisfaction, usage, and interaction with the EHR and patient portal.

CONCLUSION

Patient satisfaction is a complex variable to measure and analyze, yet it serves as an important barometer of the patient experience., The data gathered from this study provide insight into the characteristics of patient satisfaction with the patient portal while also providing a reference for future measurements. Such studies can guide the future development of policies and regulations that govern the patient portal, with the ultimate goal of increasing patient portal functionality and usability to enhance the patient experience.

PROTECTION OF HUMAN AND ANIMAL SUBJECTS

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects. Waiver of consent was granted by the IRB for this study involving no more than minimal risk and not adversely affecting the rights and the welfare of relevant individuals.

FUNDING

This project was supported by the Vanderbilt Institute for Clinical and Translational Research (VICTR), which uses CTSA award No. UL1TR002243 from the National Center for Advancing Translational Sciences. BDS was supported by the 4T15LM007450 training grant from the National Library of Medicine. Its contents are solely the responsibility of the authors and do not necessarily represent official views of the National Center for Advancing Translational Sciences or the National Institutes of Health. This project used the VUMC Research Derivative, which is supported by CTSA award No. UL1TR000445 from the National Center for Advancing Translational Sciences.

AUTHOR CONTRIBUTIONS

All listed authors contributed significantly to the study design, data acquisition, analysis, and interpretation. Each author has made substantial contributions to the drafting and revision of the manuscript as well as the final approval of the manuscript to be published. All authors will be held accountable for any aspects regarding accuracy and integrity of this work that requires investigation or resolution.

CONFLICT OF INTEREST STATEMENT

None declared.

Click here for additional data file.