Using a Patient Portal to Transmit Patient Reported Health Information into the Electronic Record: Workflow Implications and User Experience.

INTRODUCTION: This project implemented an integrated patient self-reported screening tool in a patient portal and assessed clinical workflow and user experience in primary care practices.
METHODS: An electronic health risk assessment based on the CMS Annual Wellness Visit (AWV) was developed to integrate self-reported health information into the patient's electronic health record (EHR). Patients enrolled in care coordination tested the implementation. The evaluation plan included quantitative and qualitative measures of patient adoption, provider adoption, workflow impact, financial impact, and technology impact.
FINDINGS: Seventy-two patients completed the sample AWV, and 80% of the questionnaires had clinical findings that required provider follow-up. Patients expressed satisfaction with the portal, as it enabled them to view their health record and enter information. Implementation did not reduce office staff time. Providers and office staff agreed that an electronic system for adding information to their record would increase patient satisfaction, but they expressed concern with the need to promptly review the information and the time involved to accomplish this prior to an office visit. DISCUSSION: Despite satisfaction among patients, portal adoption is still low, due to technological limitations and to the lack of adaptability to primary care practice workflow. Notwithstanding those barriers, the use of the portal for completion of repetitive tasks, such as screening tools, should be encouraged.
CONCLUSIONS: Patients can effectively use portals to complete the patient reported section of the CMS AWV. However, if the information is not completed during the same day of the office visit, the time required to address health findings outside of a regular office visit is uncompensated, and diminished the enthusiasm for this process among primary care practice staff.

Introduction

Despite the potential benefits of the use of patient portals to patients, caregivers, and providers to improve communication1 and better coordinate patient care,2 the adoption of patient portals is still limited.3,4

Health care providers, mostly in primary care, are asked to promote prevention and wellness, and to improve the quality of care while being more efficient in doing so. Many Health Information Technology (HIT) interventions, including electronic health records (EHRs) and decision tools, have been added to primary care practices to be able to accommodate all requirements including the Meaningful Use (MU) stage 2 criteria. MU stage 2 states that five percent of patients should view, download, and transmit health information,5 which necessitates many changes in work processes. HIT, such as patient portals, has the ability to allow exchange of health information between health care providers, patients, and caregivers, and also the ability to provide screening and decision tools necessary for the coordination of care. Primary care physicians may consider adopting patient portals, to improve efficiency, and in response to MU incentives6 along with requirements put forth by the Affordable Care Act.7 However, health care providers and administrators may need assistance in the implementation process of a new technology, the workflow redesign,8 and the cost necessary for the implementation.9

This project evaluated the use of a patient portal within primary care practices to transmit self-reported health information into their EHRs, and to evaluate the potential workflow implications. In order to accomplish that, the Centers for Medicare & Medicaid Services (CMS) Annual Wellness Visit (AWV)10 patient-reported requirements, including quality driven Patient-Reported Outcomes (PROs), were incorporated into the patient portal to be completed by patients before the visit, and the information from the portal was prepopulated into the EHR. The AWV allows health care providers to assess preventive measures, helping providers determine if their patients have any emergent risk factors, which allows them to focus on those factors during the wellness visit. The AWV self-reported elements of a health risk assessment (HRA)10 are suitable for the use of electronic media, such as patient portals. By using patient portals, patients can add the required reported information into their EHRs, and have the potential to not only facilitate workflow in the primary care practice but to improve their experience during the primary care visit.11 In addition, there is potential for improving compliance with coding requirements and facilitating the billing process. The evaluation also aimed to understand the barriers that limit portal adoption at nonurban and rural primary care practices by patients and providers.

Methods

This study incorporated a prospective, observational design to assess the impact of patient portals in primary care practices, when used by patients to add patient health information into the EHR. Specifically, this study looked to demonstrate the following: (1) the feasibility of using a patient portal to complete a wellness questionnaire prior to the patient’s annual visit to the primary care office, including the development of the interface between the EHR and the patient portal for patient-referred information and PROs; and (2) the implementation and evaluation of the effectiveness of use of the patient portal to that end. The interface was tested by using the CMS AWV patient-reported section. The CMS AWV was chosen because it was a relatively new process for primary care practices, it had the potential to be well received by health care providers and administrators, and it was linked to a reimbursed process.

This study was approved by an Institutional Review Board to ensure the protection of participant rights and confidentiality, and patients were enrolled after providing informed consent. The study period was from June 2013 to January 2016.

In order to assess the impact of the patient portal, the evaluation of the implementation was based on the Agency for Healthcare Research and Quality (AHRQ) HIT evaluation tool kit framework.12 This framework was designed specifically to evaluate HIT projects, and it allowed for a combination of methods that resulted in findings easy to disseminate to all potential future adopters—in this case, of patient portals as a vehicle for patients to include patient health information in the EHR. The evaluation plan included the following: (1) patient adoption measures, (2) provider adoption measures, (3) workflow impact measures, (4) financial impact measures, and (5) technology impact measures. All measures included both quantitative and qualitative data to evaluate the implementation of the CMS AWV patient-reported requirements into the portal and into the patients’ EHRs. A pre–post study design was used to assess the clinical workflow process before and after incorporating the annual wellness questionnaire into the portal. Focus groups and structured interviews were used to assess patient and provider experience during the implementation. All focus groups and structured interviews were conducted by an independent qualitative researcher not affiliated with the principal investigator’s institution.

Patient Adoption Measures

The rate of adoption was calculated as the percentage of patients among the study population who used the portal to complete the wellness questionnaire. In order to get this information, the EHR was reviewed to assess how many patients had access to a computer and an email address that allowed them to use the portal. All chronic condition patients who participated in care coordination and had an email registered in the EHR were invited to participate. Those who were interested in participating received training on how to use the portal. Patient participants who used the portal to complete the wellness questionnaire were considered users, and those who enrolled but did not use the portal to complete the wellness questionnaire were considered nonusers. To evaluate satisfaction with use of the portal to complete the wellness questionnaire, a satisfaction survey was included at the end of the questionnaire for all who used the portal. Adoption, attitude, and satisfaction were assessed qualitatively by focus groups with patients considered users, and with those patients categorized as nonusers. The themes for the focus groups with the users were the following: patient confidence in ability to use the system, patient confidence in security of the system, patient trust in accuracy of health information, general satisfaction toward the portal, and satisfaction using the portal to complete the wellness questionnaire. For those who did not use the portal to complete the wellness questionnaire, the interviews were focused on patient preferences, reasons for not using the portal, and what features they would like the portal to have to encourage them to use it.

Provider Adoption Measures

Providers’ (physicians and nurse care coordinators) adoption, knowledge, attitudes, and satisfaction with the new implementation of the wellness questionnaire into the portal were evaluated by a series of structured interviews with those providers who received a completed wellness questionnaire for review. The themes included in the interviews were knowledge of the questionnaire being implemented, time management and practice productivity, ease of use, overall satisfaction, motivation to use the portal, and computer use confidence.

Workflow Impact Measures

The workflow of incorporating the CMS AWV from the patient portal to the EHR, including PRO, was assessed by implementing a new protocol in the primary care practices. The protocol encompassed identifying the individuals in the practice who received the wellness questionnaire information for review, triaged the information, identified needs for further follow-up, communicated with patients, and linked the protocol to the current billing processes. Workflow impact measures, created to address the health care administrator and provider needs, were collected mostly by feedback from the clinical staff to the research team during regular weekly follow-up meetings. Direct (i.e., real-time) observation by research staff was not feasible since it was difficult to predict when a patient would complete the wellness questionnaire. Nevertheless, research staff did conduct a simulation with a medical assistant completing the tasks associated with the CMS AWV process to gather information on the current process before implementation. Questions to participants, patients, and staff comparing the pre- and postimplementation scenarios were included during the focus groups and interviews. Simulation measures included time required to complete the task by patients, office staff, and providers; number of staff required to complete the tasks; technology changes required to complete the tasks; and training necessary to complete the tasks. The focus groups with the nurse care coordinators and the structured interviews with providers included the effect on productivity as a workflow impact theme.

Financial Impact Measures

Financial impact measures were intended to answer the following questions:

Did the billing collection for the AWV increase after implementation? The billing staff provided a percentage of reimbursement on AWV billing for the year preceding implementation and during the year of implementation.

Did the panel size for each provider increase as a result of implementation? The potential time saved during the wellness visit could increase the number of patients seen by each provider.

Did the implementation require an increase in staff? A determination of the need to increase staff would increase expense for the primary care practices. Assessment of financial impact included a combination of workflow diagram comparisons, and results from the focus groups and structured interviews, including the effect on patient satisfaction and provider productivity.

Technology Impact Measures

Technology impact measures were based on the identification of the potential technology-induced barriers within the implementation process, with the intention to develop strategies to address limitations and barriers to use. Potential barriers were classified as the following:

Failures to log on.

Low computer literacy. A computer literacy survey was included in the wellness questionnaire to identify study participants’ comfort level in using computers.

Ease of completing the wellness questionnaire in the portal, which was included as a question in the satisfaction survey that immediately followed the wellness questionnaire.

More information was obtained during the focus groups with patients, in which two themes were included to assess the impact on technology: ease of completion via the portal, and overall portal technology.

Setting

The use of the patient portal to complete the CMS AWV was implemented in five primary care practices accredited as Patient Centered Medical Homes, which combined had over 30,000 active patients and 88 providers, including 28 residents from a family medicine residency program. Each practice had an established nurse-led care coordination program for patients with chronic conditions, which the study evaluation team targeted for enrollment. All five practices had EHR and a Patient Portal with a range of functionalities, including connection to the statewide health information exchange. Prior to the study, the portal was used by patients only for review of health care information (problem list, medication list, and laboratory and radiology reports), scheduling appointments, requesting prescription renewals, and obtaining referrals. Moreover, prior to the study the annual wellness visit was completed by a medical assistant and the provider, who completed the questionnaire with patients and entered the information into the EHR (Figure 1A).

Figure 1A

Workflow Before Implementation of the Annual Wellness Questionnaire in the Primary Care Practice

Target Population

All active care-coordinated chronic-condition patients, defined as having received care coordination services by their primary care practice, and who had internet and computer access, with an assigned email address identified in the EHR, were considered eligible and invited to participate in the project. There were no exclusions based on the patients’ previous use of the portal or any other criteria. All patients enrolled in the annual wellness questionnaire implementation process were invited to participate in focus groups, both users and nonusers of the portal. Providers, care coordinators, and office staff who reviewed the system-generated narratives of the annual wellness questionnaire implementation were also invited to participate by providing feedback on their experiences using the online questionnaire. Qualitative analysis of focus groups and structured interviews consisted of no less than ten percent of the total participant sample, including patients, providers, care coordinators, and office staff.

Technology Infrastructure

The HIT interoperability used involved a combination of patient interview software (Instant Medical History) and patient portal (Kryptiq CareCatalyst), both interfacing with the electronic health record (GE Healthcare Centricity). Information provided by patients using the portal was included as independent observational terms in the EHR, generating an easy-to-read narrative by the health care providers.

Implementation Process

The electronic survey embedded in the portal was called the “wellness questionnaire,” and was used to test patients transmitting information into their EHRs, and their experience doing it. The implementation of the electronic questionnaire also served to test health care providers and office staff reviewing and acting on the transmitted information. The wellness questionnaire included the following:

Patient demographics

Physical and emotional functional status

Falls screening

Depression screening

Mental health screening

Confidence and patient activation

Alcohol and drug use

Exercise

Nutrition

Seatbelt and helmet use

Dental health

Other assessments summarized in Table 1.

Table 1.

Assessments Included in the Wellness Questionnaire (57–70 Questions)

TOPICS	WELL-BEING FACTOR	STANDARD QUESTIONNAIRES
1. Behavioral risks	Alcohol	AUDIT
	Seat Belt	KMTHA
	Smoking Status	NIST
2. Patient-Reported Outcomes	Pain	PROMIS Q7
	Mental Health	PROMIS Q4; PHQ9
	Health Rating	EQ-VAS
	Fatigue	PROMIS Q8
	Emotional Problems	PROMIS Q10
	Self-Efficacy	CDSES
	Quality of Life	PROMIS Q2
	Physical Function	PROMIS
3. Functional ability	Mouth Problems	KMTHA
	Instrumental Daily Activities	Katz-IADLs
	Balance	KMTHA
	Falls	KMTHA
	Daily Activities	Katz-ADLs
4. Psychosocial risks	Support System	HYH, KMTHA
5. Instrumental ADLs	Financial Issues	HYH

Notes: Alcohol Use Disorders Identification Test (AUDIT); Kaiser Permanente Medicare Total Health Assessment (KMTHA) Questionnaire; How’s Your Health (HYH); National Institute of Standards and Technology (NIST); Katz activities of daily living (ADL) and instrumental activities of daily living scales (IADL)

A separate patient satisfaction survey was taken immediately after the wellness questionnaire was completed, to elicit the patients’ experience of completing it.

Incorporation of the annual wellness questionnaire into the portal necessitated an office protocol to manage the results of the questionnaire when it was received by the practice. The office protocol included an assigned triage coordinator for all the primary care practices, who was an experienced medical assistant, to triage the severity of the findings generated by the portal (Figure 2). The triage coordinator received all the results of the wellness questionnaires on her computer desktop, and it was her responsibility to send the results of the questionnaires to the appropriate provider for review and approval to be included in the EHR electronically. She also triaged clinical alerts to either the patient’s physician or the patient’s care coordinator based on the color-coded clinical alert narrative and score generated by the portal. Reports generated by the EHR and portal systems enabled tracking of participant log-ons and data transferred to the EHR, as well as the number of providers who reviewed or signed off on clinical narratives. To develop the clinical alerts, all questions from the wellness questionnaire were weighted by clinical importance by a team comprises of physician leaders from the practices and nurse care coordinators. All responses programmed in the portal system were color coded in red, blue, or black fonts. Black font in the narrative indicated that there were no positive screening questions, and there was no need to follow up. Blue font in the narrative indicated the wellness questionnaire had some answers that were not urgent but required follow-up with the patient; clarification of the findings could be reviewed within 48–72 hours. Red font in the narrative indicated that the patient needed to be contacted within the next 24 hours for clarification of the questionnaire responses.

Figure 2.

Clinical Alert Narrative Generated in the EHR after Wellness Questionnaire Completion

Results

Interface Development

This study included the development of the interface between the EHR and the patient portal, with patient-referred information and PROs from a CMS-based annual wellness visit questionnaire. The internal information systems (IS) team working in collaboration with the IT vendors was able to create interfaces that directly linked the front-end forms filled by the patient with the observation terms in the EHR. This functionality enabled the practices to automate the electronic documentation process of the completed wellness questionnaires by the medical assistants and office staff as described above. Successful development of the interface (i.e., transfer of data from the wellness questionnaire to observation terms in the EHR) took approximately six months.

During testing of the interface, two primary limitations were identified:

Limited compatibility with internet browsers. In this case, only Microsoft Internet Explorer (version 11 and above), Safari, Google Chrome, and Mozilla Firefox were able to complete the task.

Software updates, which interrupted the interface. After software updates by any of the vendors, the interface was broken, with the need to troubleshoot. Each time this happened, restoration of the interface required a coordinated effort between the internal IS department and the outside vendor. Each incident took up to three months to restore functionality, during which patients were not able to use the portal.

Implementation of Portals to Incorporate Patient Self-Reported Health Information into the HER

After the interface was built between the portal and the EHR, the evaluation of the implementation was started by recruiting patients to participate in the study, and identifying the providers, nurse care coordinators, and office staff involved in the implementation process. The evaluation results were reported as patient experience, providers’ experience, workflow implications, financial implications, and technology issues.

Patient Experience

Ninety-two chronic-condition patients were enrolled and trained to use the portal; 70 percent of them had never used the portal. Seventy-two participants (78 percent) completed at least one wellness questionnaire via the patient portal and were classified as users of the electronic wellness questionnaire. Twenty patients (22 percent) did not complete a wellness questionnaire and were classified as nonusers. Forty-eight participants (52 percent) were followed up for more than 12 months and were able to complete and assess the workflow process for a second time (Figure 3). Three-quarters of the participants were female with a mean age of 59.6 ± 12.7 years (Table 2), and more than 88 percent of them reported using computers for banking (Table 3). Based on the satisfaction survey completed after the wellness questionnaire, patients expressed high levels of satisfaction after the use of the portal: ease of use (93 percent), type of questions (83 percent), and total number of questions (81 percent).

Figure 3.

Patient Enrollment

Table 2.

Demographic Characteristics of Patient Users of the Portal (N= 72)

Females/Male (%)	72/28
Age (Mean ± SD)	59.6 ± 12.7
Medicare Part B (%)	48.6
Married (%)	50
Smokers (%)	52.8
Retired (%)	55.6

Table 3.

Computer Literacy among Patient Users of the Portal (N=72)

IN THE LAST 12 MONTHS…	% USERS
1. Have you used a computer to seek information online?	86.1
2. Have you used a computer to access and use a patient web portal or electronic health records at a doctor’s office?	59.7
3. Have you used a computer to conduct online banking and/or make purchases online?	88.9
4. Have you used a computer to communicate with family and/or friends?	95.8
5. Have you used a computer for entertainment or information?	62.5
6. Have you used a computer to create and maintain a personal medical record?	29.2
7. Have you used a computer for other functions (games, school, work)?	4.2

All patients using the portal were contacted and invited to participate in a focus group. One two-hour focus group was conducted with 11 patients (15 percent) considered users. All patients in the focus group had completed at least one wellness questionnaire and some had been in the study long enough to have taken it twice. Patients were asked to comment on the following themes: ease of completion via the portal, effect on patient care, effect on patient satisfaction, effect on productivity, other perceptions, and portal technology. Patient users identified patient-related factors such as confidence in their ability to use the technology, security concerns, and availability of effective training to use the portal to be associated with their adoption. Patients expressed a range in their trust of the data security within the portal to keep their health information secure from data breaches. Some reported feeling confident enough to use the portal in public settings, while others felt that was a risk they would never take. A few expressed doubts about the security of the system based on personal or professional experiences. Patients also identified provider-related factors, such as having providers who did not recommend technology use. Some system-related issues that could affect adoption were identified by patient users as logging on to the portal and organization of the portal features. Overall, patients thought the new process would improve the care they received because their doctor would have more time to review information prior to their office visit. A summary of findings from the focus group is included in Table 4.

Table 4.

The Annual Wellness Questionnaire via the Portal: Patient, Provider, and Care Coordinator Perceptions

THEMES	PATIENTS	PROVIDERS	CARE COORDINATORS
Ease of completion via the portal	Some survey questions were perceived as ambiguous and provided limited response choices.	Not applicable	Not applicable
Effect on patient care	May improve the care they receive because their doctor will have more time to review information prior to their office visit.	No comments	No comments
Effect on patient satisfaction	Pleased with the rapid response time and the level of support from their physician and care coordinator.	Probably increases patient satisfaction because it enables patients to have direct communication with their physician and provides another access point to the practice.	Potential to increase communication and access for patients, and subsequently to increase patient satisfaction by being able to address health concerns in a timely manner.
Effect on productivity	Probably will not save time for providers.	Concerns related to uncompensated burden on their time, when receiving alerts from the wellness questionnaire outside of the regular office visit. Little confidence that the survey will streamline the office visit workflow.	Time required to screen the results of the wellness questionnaire ranged from 15 minutes to 2 hours, depending on the issues. Information received increased the number of phone calls they made to patients.
Other perceptions	Mixed feelings about trusting the security of the portal, and concern that the survey was collecting health information as part of the ACA.	The wellness questionnaire alerts were handled mostly by the care coordinators.	The wellness questionnaire was less related to their duties and more related to the physicians’ duties and the annual visit requirements.
Portal technology	Technical aspects of the portal worked well, with some exceptions: log on errors and password issues.	Need for training and technical support, and lack of reliability	Not completely satisfied due to portal technical difficulties in sending information to patients.

Care Coordinator Experience

Ten care coordinators participated in the project. They followed most of the blue alerts (61 percent) identified by the electronic system in the EHR. Only 8 of the 10 care coordinators had patients who completed the electronic wellness questionnaire. All 10 care coordinators (100 percent) participated in a one-hour focus group discussing the following themes: effect on patient care, effect on patient satisfaction, effect on productivity, other perceptions, and portal technology. During the focus group, they were asked to describe a typical workflow for the wellness questionnaire based on their user role and responsibilities. They were asked about costs in terms of time to process the questionnaire responses, when they processed the questionnaires during their workday, and their perceptions about the value of the wellness questionnaire information in improving the care of their patients. They were also prompted to describe the specific steps taken if a questionnaire response indicated acute (red) or concerning (blue) clinical alerts, and their level of confidence in the response protocol. Care coordinators reported that it typically took about 15–20 minutes to screen an average survey but more complex issues could take up to 2 hours to resolve, and that they only assessed the EHR of patients they were actively coordinating. A summary of findings are included in Table 4.

Health Care Provider Experience

The five piloted primary care practices had a total of 89 providers. Only 24 health care providers, including physicians and nurse practitioners, had to review a system-generated narrative and sign off on the annual wellness visit questionnaires through the portal. After the providers reviewed a narrative, they were able to electronically confirm orders and consult patients to prepare a personalized health plan. All 24 providers were invited to participate in a structured interview. Five providers couldn’t be contacted, as they had left the practices. Of the 19 providers remaining, 5 accepted the invitation (26 percent). Interviews of the providers revealed that most of the time it only took a few minutes to sign off on the survey if the care coordinator had already reviewed it for concerning information. The biggest concern that the providers had was the timing of the completion of the wellness questionnaire by the patient.

The wellness survey would be very useful if it is done right before a Medicare wellness visit. Medicare suggests that the wellness survey be offered online so the patients can do it when they can think about it at home. When it is done out of the scheduled visit, and patients indicate they are depressed, I don’t know what I am supposed to do with that information…From my standpoint, if it is given in the right timing where we can act on it, that is good but I think that just having the wellness survey not associated with a visit is a problem…It is not a problem with the portal but how the portal is managed on our end that is the problem.

Providers were asked to evaluate the portal in its ability to increase practice productivity and patient satisfaction and to decrease practice costs (i.e., time and staff resources). One physician believed that the portal will eventually increase practice productivity by decreasing the amount of personal time needed to respond to patient care needs. Other providers interviewed reported feelings of decreased productivity due to the number of emails they needed to respond to after regular patient visit hours. These providers felt that because they could not be compensated for most of the time they were responding to patient emails, the practice was actually reducing its productivity by treating patients during nonreimbursable time. All five interviews were completed by the unbiased qualitative researcher, and they were done by telephone—lasting an average of 20 minutes each. Providers were approached about their perceptions of the following themes: effect on patient care, effect on patient satisfaction, effect on productivity, other perceptions, and portal technology. A summary of provider perception is described in Table 4.

Workflow Implications

One hundred twenty questionnaires were triaged and addressed by the office staff, care coordinator, or provider, following protocol. Based on the patient responses to the questionnaire, the system created a narrative that was reviewed by the provider. The narrative highlighted any tested well-being factors for which the patient was at risk. Based on the response severity, the risk categories were blue (moderate) risk or red (high) risk (Figure 4). From the 120 questionnaires received by the primary care practices, 104 had some clinical finding; 85 (82 percent) had more than one alert; 60 (58 percent) had 3 or more alerts. The number of findings ranged from 1 to 13 per questionnaire, with an average of 3.9 alerts per questionnaire received (Figure 2). In the sample analyzed, 54 percent of the questionnaires had red alerts, and more than 80 percent had some clinical finding that required provider or care coordinator follow-up. Among 403 alerts identified, 23 percent were red alerts requiring the providers to contact patients for clarification, and 77 percent were blue alerts considered a concerning matter, to be reviewed with the patient but not as an urgent matter. During the implementation, it was found that not all of the clinical alerts needed attention, mostly because they were already part of the patients’ problem list and/or plan of care. Red alerts were followed up by the physicians in 61 percent of the cases, and blue alerts in 39 percent. The most common clinical alerts were associated with mental health through Patient Health Questionnaire-9 (PHQ-9), physical function, fall risk, and pain. Red alerts were associated only with depression screening, pain screening, and alcohol screening (Figure 4).

Figure 4.

Clinical Findings from the Wellness Questionnaire: Frequency of Blue and Red Alerts

Based on the experience with the 120 questionnaires received by the practices, two workflow diagrams were prepared representing workflow before (Figure 1A) and after implementation (Figure 1B). When compared, the workflow after implementation showed the following: (1) a reduction of time spent by medical assistants during the office visit from 10–25 minutes to 5–10 minutes; (2) no significant changes in the time spent by providers during the office visit—in both cases providers spent between 10–15 minutes; and (3) an increase of time spent by the office staff, care coordinators, and providers prior to the office visit in addressing the findings of the wellness questionnaire, which took between 10 minutes to 2 hours depending on the severity of the identified issues. As part of the study, patients were encouraged to complete the wellness questionnaire as a way to evaluate their patient experience. Consequently, many of the questionnaires were received by the practices on dates not associated with a regular annual office visit for those patients; nevertheless, the practice was responsible to follow up on the findings.

Figure 1B

Future Workflow for Implementing the Annual Wellness Questionnaire in the Primary Care Practice

Based on focus groups and interviews with clinical staff, there seemed to be consensus agreement among the providers and office staff that automation of the process will reduce the time taken to complete the clinical visit. However, during focus groups the care coordinators reported that the time required to screen the results of the wellness questionnaire, as mentioned above, ranged between 15 minutes to 2 hours depending on the health issues of a specific patient. Information received sometimes increased the number of phone calls they made to patients. Although this finding was expected, the disruption of a regular office day was significant to care coordinators and providers. Care coordinators expressed that the required follow-up action ranged from a phone call, in almost 95 percent of the cases, an office visit or specialist referral (approximately 5 percent of the cases), and in one case a hospital admission.

The users’ adoption, attitudes, and satisfaction are summarized in Table 4.

Technology Issues

Of the 72 users of the portal who completed at least one wellness questionnaire, 11 (15 percent) reported technology issues including the following: responses where the information from the portal was not able to be transferred to the EHR (8 users); problems using the internet browser; and difficulty logging on to the portal (3 users). Among the nonusers group, only one reported technical difficulties as being difficulty logging on to her portal with many attempts at resetting her password. At the end of the study, all data from 120 completed surveys were flowing smoothly from the forms filled by the patients to the observation terms in the EHR. Providers felt there were underlying issues related to troubleshooting and the need for technical support, as well as the burden it placed on the practice professionals to address secure emails and questionnaire responses in an appropriate and timely manner.

Patients who used the portal reported technical challenges associated with barriers to logging on or gaining access to the portal on a consistent basis. Many of these were attributed to the time that was required to restore the system after recent software updates. In some cases, patients’ passwords needed to be reset; in others, there were problems with the actual link of the patient log to the patients’ provider due to provider turnover. Moreover, patients perceived that the portal was omitting important information from their EHR because the patient could only be linked with a single provider when, in reality, in a large health care system, patients may be associated with many different practices and providers.

Financial Impact

Whether the implementation of this new process had an impact on financial measures—such as an increase in the number of eligible patients receiving AWV, increase in the provider and office staff time saved, and increased revenue—can be assessed by comprehensive collection and analysis of data over time; however, these two initial observations did not answer those questions. When billing reports prior to implementation were reviewed, they showed a high compliance rate of potential patients completing the CMS AWV. Therefore, the new process would not necessarily increase compliance rate within the piloted primary care practices participating in the study. Moreover, when the time for providers to review the AWV results was observed and compared with the time on the new process, there were no significant changes. In terms of the staff needed to implement the CMS AWV, if the AWV questionnaire is completed on the day of the patient’s scheduled visit, there will be no need to have a triage coordinator. In addition, the electronic questionnaire process will not affect practice workflow since any issues identified in the questionnaire will be immediately addressed by the provider during the office visit.

Portal Nonuser Adoption and Use Barriers

All nonusers (n=20) were contacted to participate in a focus group. Only three (15 percent) agreed to participate. The focus group with the nonusers revealed the following:

Patient preference for face-to-face communication with their provider. Email communication was not viewed as an attractive alternate method of communication. All nonusers of the portal expressed a strong preference for personal contact, either as an office visit or a phone conversation.

Patient’s lack of trust in the accuracy of health information. The nonusers group did not have adequate understanding of how the portal would be able to provide health records for their review. All focus group members had experiences with inaccurate health information in their records, and felt the best way to review and verify their information was to ask for a paper copy and work with their provider or care coordinator to make the corrections.

Patient’s lack of trust in security of health information. Two of the three members of the focus group expressed concerns about the security of their personal data and sensitive health information if they were to use the portal.

Improved training features and support offerings would be beneficial. Offering a hands-on training class or online videos might increase interest in the portal for people who were not confident in their computer skills but wanted to use the portal to manage their health information.

Limited functionality with certain internet browsers.

Assistive technology features for literacy and visual impairment support. An assistive audio feature developed to read reports or messages from providers could help some patients overcome accessibility-related barriers to using the portal.

Conclusion

Current portal technology allows for the upload of an electronic health screening questionnaire, the results of which integrate into a patient’s EHR upon completion, with an easy dashboard for providers to assess the patient’s potential risks.

The perceptions of portal usability varied among users and seemed to be the result of differences in the roles and responsibilities of the groups studied. On the one hand, with the exception of technical difficulties accessing the portal, patients expressed satisfaction using the portal to complete the annual wellness questionnaire and were pleased with the rapid response time by the practice in addressing their health concerns, but they acknowledged that it probably would not save time for their providers. On the other hand, providers (physicians and care coordinators) did not completely embrace the use of the portal due to the following: (1) the perception that it did not improve productivity measures, i.e., it increased the burden on their time but did not provide compensation for that time; (2) the limitations of the technology itself requiring maintenance, training, and help desk support; and (3) perceived limited relevance to direct care. Most of the required follow-up from the wellness questionnaire was related to depression screening, pain management, fall risk, and physical function limitations, which not only added extra time to assess but were difficult to assess when patients were not physically present. Providers recommended the use of the portal to complete the self-reported assessments when the assessment is completed on the same day as an office visit—to be able to address health care findings during the visit when they can talk to the patient and see any potential signs of a problem.

Overall, portal technology still has barriers related to interface compatibility. However, as the technology improves, portals may facilitate the process for patients to incorporate health-related information into the EHR, as has been demonstrated for the annual wellness visits in this study.

Discussion

The adoption of patient portals should be seen, not as a burden, but as a benefit to the primary care practices, not only in terms of facilitating current workflow, but also by improving the experience between providers, patients, and caregivers.

Health care providers’ interest in using HIT, particularly patient portals, is essential to the adoption and usefulness of web-based portals. This study identified some contributing factors to low adoption by providers, such as technical difficulties, the management of patient-reported data, the lack of understanding of their patients’ satisfaction with the portal, and the implicit workflow changes requiring more staff support. It was also found that adding productivity measures to the review and verification of patient responses to screening tools, outside of the office visit, has the potential to improve the adoption of patient portals at primary care practices.

Technology difficulties are still a limiting factor in the adoption of HIT among patients, physicians, and care coordinators. Constant EHR updates and portal software updates were serious limitations in this project, requiring the revision of portal interconnectivity with the EHR after every software update. Another limitation was the lack of compatibility of the portal with all internet browsers, requiring users to download and use a different internet browser in order to access the patient portal. Other studies have reported similar findings,17 which suggests that portal vendors need to respond more rapidly to the constantly changing health care environment. Technical difficulties due to vendor capabilities, integration within the EHR, and interfaces with survey products should not be underestimated.

Although the importance of capturing PROs is recognized, many health care professionals remain uncertain about incorporating this data into practice and need opportunities to lend their input and expertise to determine its utility.18 PROs have the potential to improve communication between patients and providers through assessing the effect of the changes after any plan of care modification on functional status, quality of life, and satisfaction. Most recently, research focused on patient-centered outcomes and the technology needed to enable patients to report them, but this has not led to increased implementation. The lack of provider feedback is probably the reason why the implementation is not more disseminated. There is not currently an evidence-based protocol that allows primary care providers to make changes in their management due to changes in PRO metrics. More studies are needed to clarify the benefits of PRO for a primary care practitioner who still needs to handle the pressure of effective, productive work. Moreover, more studies are needed to clarify the benefits of PRO for a primary care practitioner during routine annual visits.

Overwhelmingly, all the user groups, patients, providers, and care coordinators agreed that, when the portal was working well, its greatest strength was in providing increased communication and access for managed care patients. This was true even though some of the providers and care coordinators were unhappy about its impact on their own duties. Providers also believed in the benefits of collecting data about how the portal could potentially improve patient satisfaction and address patient health concerns in a timely manner, but were unaware of the enthusiastic use by patients who would like to see even more capabilities added to the portal. Patients reported the portal’s primary strength was its ability to provide a quick and flexible mode of communication with their provider in cases of minor health concerns, whether for themselves as patients or in their role as a caregiver. This project identified a misperception among providers regarding their patients’ satisfaction with using the portal.

Workflow also affects the adoption among providers. In this study, clinical alerts prompted by the wellness questionnaire were not necessarily associated with the patient’s current problem list. Therefore, some clinical alerts were not considered as relevant or needing review, as they were issues already addressed by the providers in previous visits. That led to the conclusion that the use of the portal for completion of screening tools in close proximity to the patient’s next office visit should be encouraged in order to reduce interference with the office workflow, allowing time for providers and other office staff to review the information and reduce errors as the information is verified during the office visit intake.

Although substantial effort was made to collect and analyze as many portal stakeholder experiences as possible, conclusions based on the qualitative data alone are limited by the small size of each user group.

The results of the evaluation are of interest to different stakeholders, including health care practitioners and administrators who are considering implementing HIT to comply with the MU criteria. This evaluation of the implementation of a web portal for patients to enter self-referred information into the EHR demonstrated some of the barriers to implementation—from the patient, provider, and office staff perspectives. These include the need for technical support and troubleshooting capacity, patient and provider training, and the importance of considering the best time for the information to be reviewed by patients and providers in order to be clinically relevant and actionable.

The information gathered in this study provides useful insights for current and future HIT adopters and may improve the adoption rate. This study led to a subsequent analysis by the authors on the value of the portal to patients’ experience, self-efficacy, and perception of quality of care, when used as a tool for communication between patients and health care providers.

Most of the burden that primary care providers experience may be attributable to the increased number of preventive measures that are required to be captured in a visit. The role of HIT to help adjust the burden may be more effective when future studies can confirm the real benefit of the extra data collected, the time when the data should be collected, and the appropriate staff to review the data. For example, more studies may be needed to assess the current impact of the CMS AWV questions among primary care practices, and their relevance to patient care.