Clinical Use of an Electronic Pre-Visit Questionnaire Soliciting Patient Visit Goals and Interim History: A Retrospective Comparison Between Safety-net and Non-Safety-net Clinics.

INTRODUCTION/
OBJECTIVES: We examined an initial step towards co-generation of clinic notes by inviting patients to complete a pre-visit questionnaire that could be inserted into clinic notes by providers and describe the experience in a safety-net and non-safety-net clinic.
METHODS: We sent an electronic pre-visit questionnaire on visit goals and interim history to patients at a safety-net clinic and a non-safety-net clinic before clinic visits. We compared questionnaire utilization between clinics during a one-year period and performed a chart review of a sample of patients to examine demographics, content and usage of patient responses to the questionnaire.
RESULTS: While use was low in both clinics, it was lower in the safety-net clinic (3%) compared to the non-safety-net clinic (10%). We reviewed a sample of respondents and found they were more likely to be White compared to the overall clinic populations (p < 0.05). There were no statistically significant differences in patient-typed notes (word count and number of visit goals) between the safety-net and non-safety-net samples however, patients at the safety-net clinic were less likely to have all of their goals addressed within the PCP documentation, compared to the non-safety-net clinic.
CONCLUSIONS: Given potential benefits of this questionnaire as a communication tool, addressing barriers to use of technology among vulnerable patients is needed, including access to devices and internet, and support from caregivers or culturally concordant peer navigators.

Collaboration between patients and providers is a critical component of patient-centered care and is associated with improved health outcomes.[1-3] Open-ended questions at the start of a visit are associated with increased patient satisfaction, and in a randomized trial, pre-visit questionnaires administered via electronic portals assisted patients in preparing for visits and were viewed favorably by providers.[4,5] An innovative way to strengthen patient-provider collaboration is allowing patients to co-write their clinic visit notes.[6,7] An early study of patients and providers co-writing clinic visit notes involved patients typing their visit agendas prior to the clinic visit and inserting this text into the body of the doctor's note. Patients and providers involved in these co-written notes perceived improved communication, suggesting that co-written notes could improve patient engagement, decrease the documentation burden for providers, and improve the efficiency of clinic visits. Giving patients the opportunity to tell their story in their own words and including this in the clinic note is empowering, and may particularly appeal to patients who have experienced systemic oppression or disenfranchisement.

Historically, medical records have been easily available only to medical providers. HIPAA gave patients legal access to their medical records but in practice the records could still be difficult and costly to procure.[9,10] Over the last decade the OpenNotes movement has advocated making visit notes and medical records easily accessible by patients to improve transparency and collaboration. This movement is supported by evidence that patients who utilize health information technology to collaborate with their provider experience greater satisfaction with care[6,8,11,12] and improved quality of care. With implementation of the Final Rule of the 21st Century Cures Act, patients in the U.S.A now increasingly have ready electronic access to their health information.

Medical record electronic patient portals may improve patient experience or outcomes,[3,15,16] however there is concern that many patients may be unable to engage with the patient portal because of limited computer and health literacy and other barriers.[17,18] Evidence suggests that non-white patients are less likely to access their health information online[13,19-21] and are less likely to be offered access to this information. As a result, patients in some minority groups may be excluded from the potential benefits of health information technology.

Given the potential benefits of co-generated clinic notes, we wanted to understand utilization of this novel questionnaire by safety net and non-safety-net populations and had the opportunity to do so in a health system using the same technologic process and health record. We compared implementation and utilization of this novel questionnaire in two large academic clinics, one serving a safety-net population and the other a non-safety-net population. Additionally, we performed a chart review of a sample of respondents’ notes in each clinic, examining thematic content and provider use of the patient responses.

Methods

Two large academic internal medicine primary care clinics in [Seattle, WA] participated in this study between June 1, 2018 and June 30, 2019. One clinic is located at [Harborview Medical Center], a county safety-net hospital serving those with financial barriers to care and individuals from historically disenfranchised populations (the safety-net clinic). The other clinic is located at the [University of Washington Medical Center] and serves a primarily privately insured population (hereafter referred to as the non-safety-net clinic).

To be eligible, patients at these clinics had to be 18 years or older, have an active patient portal account, be able to complete the questionnaire in English, and have had three or more visits within the health system in the last 12 months. Clinic medical directors provided information about the pre-visit questionnaire to providers at clinic meetings and via email; patients did not receive any messaging beyond receiving the pre-visit questionnaire notification email. Institutional Review Board approval for the study was obtained from the University of Washington.

Eligible patients were sent an email up to seven days prior to a scheduled appointment notifying them that a new message was available on the patient portal. To answer the questionnaire, patients clicked the link in their email and logged into their patient portal account. The questionnaire asked them to type their interval histories since the most recent clinic visit, and outline their goals for the clinic visit (see Box 1 for full questionnaire). Each question had a character-limited text box for patients to type in their responses. Providers could read the responses in the patient's chart prior to and during the appointment. In the visit encounter in EpicCare (Epic Systems Corporation, Verona, WI), providers clicked a blue bar above the note labeled “Patient answers are available” to see the questions and patient-typed responses. When selected, a “Yes” button below the text titled “Do you want to add the following patient answers to your note” added the patient-entered information to the visit note at the location of the cursor in the note.

Two questions comprising the pre-visit questionnaire: interim history and goals.

Please tell your doctor how have you been since last visit. (You may want to include changes in symptoms affecting your health, in the way you are taking medications, in your habits, or other changes in your life. To help guide what you write, we suggest you also read your doctor's last note, focusing in particular on previous plans or recommendations).

What issues would you like to focus on at your visit? (We suggest you limit what follows to the 2 or 3 things on your mind that are most important to you).

*Responses to each question were limited to 197 words due to electronic medical record functionality.

We conducted a review of a purposive sample of patient charts and associated responses to the pre-visit questionnaire from providers who had received at least three completed pre-visit questionnaires (34 providers at the safety-net clinic and 52 at the non-safety-net clinic). Using a random number generator, we randomly selected two patients from each of these providers whose charts and responses we would review, for a total of 172 patients and their visit notes.

First, four authors (HS, EM, JJ, SJ) reviewed 10% of responses to determine the main content categories in the interval history and reached consensus on the following categories: physical symptoms, medications (prescription or over the counter medications/supplements), life changes, minimal response (eg, “fine”), mental health issues (such as depression, anxiety, or panic, or medication management of mental health diagnoses), and social issues (eg housing, financial challenges, employment, disability, violence/abuse, or paperwork related to these; Table 2). Second, two co-authors (EM and HS) each reviewed all remaining responses to categorize the interval history by content. Agreement of the content categorization was assessed using a Kappa agreement statistic; we had good agreement for physical symptoms κ = 0.75 (95% CI, 0.65 to 0.86), mental health issues κ = 0.67 (95% CI, 0.48 to 0.86), social issues κ = 0.70 (95% CI, 0.43 to 0.98), and minimal response, κ = 0.79 (95% CI, 0.68 to 0.91), and very good agreement for medications κ = 0.95 (95% CI, 0.90 to 1.00), and life changes κ = 0.82 (95% CI, 0.69 to 0.94). We reached consensus for any disagreements through a consensus meeting of co-authors.

Table 2.

Characteristics of Sample of Patient Responses to the Pre-Visit Questionnaire in a Safety-net and a Non-Safety-net Clinic.

	Safety-net Clinic		Non-safety-net Clinic
	N (min, max)	%	N (min, max)	%	p-value
SAMPLE QUESTIONNAIRE DATA
Questionnaires reviewed for sample*	68		104
Questionnaires with both questions completed	61	90%	90	87%	0.55
Total # primary care providers involved in the care of the respondents	34		52
QUESTION 1 - INTERIM HISTORY
Patient wrote interim history†	66	97%	95	91%	0.20
Average word count	34 (0, 177)		29 (0, 197)		0.44
Themed content of interval history‡
Report symptoms	42	64%	59	62%	0.84
Medication related	22	33%	23	24%	0.20
Life changes/habits	11	17%	18	19%	0.71
Minimal response	16	24%	19	20%	0.52
Mental health issues	8	12%	9	9%	0.59
Social issues	3	5%	4	4%	1.00a
Number of OLD CARTS history elements in interim history§
Zero	33	50%	45	47%	0.18a
1–3	28	42%	48	51%
4 +	5	8%	2	2%
QUESTION 2 - VISIT GOALS
Patient wrote one or more visit goals‖	63	93%	99	95%
Average word count	20 (0, 181)		19 (0, 197)		0.97
Average number of visit goals in Q2	2.03 (0, 6)		1.75 (0, 5)		0.20
Primary Care Provider Use of Patient responses (Interim History and Goals)
Patient completed questionnaire and attended the visit #	56		97
PCP added patient's text to the visit note¶
Yes	39	70%	60	62%	0.33
No	17	30%	37	38%
PCP documentation addressed all of patient's goals for the visit (Q2)
Yes	38	68%	81	84%	0.0025
No	18	32%	16	16%

*Number of interim histories (Q1) left blank = 2 (safety-net clinic), 9 (non-safety-net clinic). Number of visit goals (Q2) left blank = 5 (safety-net clinic), 5 (non-safety-net clinic).

†Some responses were left blank by patients.

Could be more than one per questionnaire.

OLD CARTS stands for onset, location, duration, characteristics, aggravating/alleviating factors, radiation, timing, and severity and represent descriptions that patients could provide in their questionnaire to better characterize their symptoms or illness. There are a total of 8 OLD CARTS criteria and a patient could have provided no OLD CARTS descriptions in their questionnaire responses or up to 8. Patients answers were categorized as zero, 1–3, or 4 + , indicating no OLD CARTS descriptors were included in questionnaire responses, at least one or up to three OLD CARTS descriptors were included in questionnaire responses, and 4 or greater (up to 8) OLD CARTS descriptors were included in questionnaire responses.

Some responses were left blank by patients.

Both interim history (Q1) and visit goals (Q2) included.

Twelve patients at the safety-net-clinic and 7 at the non-safety-net clinic who completed a pre-visit questionnaire did not attend their appointment; thus, the denominators are 56 at the safety-net and 97 at the non-safety-net clinics for day of visit metrics.

Fisher's Exact test.

One co-author (EM) counted the number of clinical history elements in the interval history (see Table 2),[22,23] the number of unique goals, if included, and reviewed patient charts to obtain patient demographic information, identify whether the patient attended the visit, and if the pre-visit questionnaire responses were integrated into visit notes by the primary care provider. When patients typed specific goals for the visit, the note was reviewed to identify how often the provider documented addressing the patient goals.

Differences in proportions between groups were evaluated using chi-squared tests or Fisher's Exact tests when frequencies were low. Differences in continuous measures were evaluated using two-sample t tests. All tests were two-sided and statistical significance was evaluated at p < 0.05. Statistical analyses were performed using SAS 9.4 (SAS Institute, Cary, NC).

Results

We sent pre-visit questionnaires before 17 844 visits at the safety-net clinic and 17 590 visits at the non-safety-net clinic; three percent were completed at the safety-net clinic (n = 473) and 10% at the non-safety-net clinic (n = 1808). We sampled 172 for inclusion in the analysis.

Among 172 sampled patient respondents, a higher percentage at the safety-net clinic were white and English speaking compared to the overall safety-net clinic population (p < 0.0001); the same was true at the non-safety-net clinic (race, p = 0.03 and English speaking, p = 0.02; Table 1). Twelve (18%) participants in the safety-net sample were black compared to five (5%) in the non-safety-net sample; and one (1%) participant was Asian in the safety-net clinic sample compared to nine (9%) at the non-safety-net clinic. The participant samples from each clinic differed in terms of age ( ≥ 45 years, 79% at safety-net and 66% non-safety-net, p = 0.001) and primary insurance provider type (private insurance 21% in safety-net, 47% non-safety-net, p = 0.0001; Table 1)

Table 1.

Demographics of Patients Attending the Safety-Net and Non-Safety-net Clinics and of Each Clinic’s Respective Sample*.

	Safety-Net Clinic			Non-safety-net clinic
	Participant sample^ (n = 68)	Overall Clinic Population# (n = 3887)	p-value comparing sample to Overall	Participant sample^ (n = 104)	Overall Clinic Population# (n = 8480)	p-value comparing sample to Overall	p-value comparing Participant samples
AGE
18–44	14 (21%)	766 (20%)	0.91	35 (34%)	2659 (31%)	0.08	0.001
45–64	33 (48%)	1824 (47%)		23 (22%)	2715 (32%)
≥ 65	21 (31%)	1295 (33%)		46 (44%)	3106 (37%)
SEX
Female	34 (50%)	1747 (45%)	0.41	46 (44%)	3896 (46%)	0.73	0.46
Male	34 (50%)	2140 (55%)		58 (56%)	4583 (54%)
PRIMARY LANGUAGE
English	68 (100%)	3011 (77%)	<.0001	104 (100%)	8075 (95%)	0.02b	-
Other	0 (0%)	876 (23%)		0 (0%)	405 (5%)
RACE/ETHNICITY
White	51 (75%)	1494 (38%)	<.0001a	83 (80%)	5956 (70%)	0.03a	0.46a
Black or African American	12 (18%)	1176 (30%)		5 (5%)	461 (5%)
Hispanic	0 (0%)	525 (13%)		0 (0%)	307 (4%)
Asian	1 (1%)	440 (11%)		9 (9%)	924 (11%)
American Indian or Alaska Native	1 (1%)	50 (1%)		1 (1%)	41 (0.5%)
Native Hawaiian or Other Pacific Islander	1 (1%)	49 (1%)		1 (1%)	39 (0.5%)
Multiple Races	0 (0%)	31 (1%)		0 (0%)	86 (1%)
Declined to Answer/ Unavailable/Unknown	2 (3%)	122 (3%)		5 (5%)	666 (8%)
PRIMARY INSURANCE PROVIDER
Private	14 (21%)	622 (16%)	0.35	49 (47%)	4516 (53%)	0.44	0.0001b
Medicaid	21 (31%)	1085 (28%)		13 (12%)	856 (10%)
Medicare	30 (44%)	1712 (44%)		42 (40%)	2976 (35%)
Uninsured	3 (4%)	425 (11%)		0 (0%)	52 (1%)
Other	0 (0%)	43 (1%)		0 (0%)	80 (1%)

a. Race/Ethnicity groups were collapsed into White versus Other for statistical testing due to low numbers.

b. Fisher's Exact test.

A purposive sample of patients who completed the pre-visit questionnaire and were randomly selected from providers who had received at least 3 completed pre-visit questionnaires.

* Age, sex, language, race/ethnicity, insurance from administrative data. Percentages for each demographic category may not add to 100% due to rounding.

Clinic visit data for all patients with one or more clinic visits during the one-year study period.

We reviewed 172 patient responses to the pre-visit questionnaire (68 safety-net and 104 non-safety-net; Table 2). Of these, 10 patients answered only the history item, 11 answered only the goals question, and 151 answered both. The average length of the interval history responses was 34 words at the safety-net clinic (median: 18.50, standard deviation: 41.96) and 29 words at the non-safety-net clinic (median: 14, standard deviation: 37.71). Responses to the interval history in both clinics were most often about physical symptoms, followed by a medical history update, life changes, mental health issues, and social issues. At least half of the interim history responses included one or more history elements. For the visit goals, patients from both clinics wrote approximately 20 words in their responses (safety-net clinic: median: 6, standard deviation: 36.2; non-safety-net clinic: median: 10, standard deviation: 29.1) and had an average of two goals for the clinic visit (safety-net clinic: median: 2, standard deviation: 1.5; non-safety-net clinic: median: 1, standard deviation: 1.1). There were no statistically significant differences in the patient-typed content between the two clinics (p > 0.05). Tables 3 and 4 show example patient responses to the pre-visit questionnaire.

Table 3.

Interim History Examples of Each Theme.

Theme	Example quote from interim history
Report symptoms	Lower back pain has gotten better, middle back feels out of alignment quite often. New symptoms of rib pain in past week. (safety-net)
	coughing and lots and I mean lots of flim to the point I throw up and getting worse as each day goes by also it is causing me to have short breaths hard to breath with the flim (non-safety-net)
	Good energy but blood pressure fluctuations (non-safety-net)
Medication related	I’m Nauseated from the meds. (safety-net)
	I stopped taking the victoza and am taking my Levemir, 30 units in the morning as well as 30 units in the evening. (safety-net)
	I am taking Daily Essential Nutrients vitamins from Canada and am reducing my regular psychiatric medications as prescribed by XX. (non-safety-net)
Life changes/habits	I've had more fright so that makes me more sedentary. (safety-net)
	I need to lose more weight and I'm having difficulties eating balanced meals and losing weight while trying to get in enough food to take the vitamins I am taking. (non-safety-net)
Minimal response	About the same as before (safety-net)
	I've been feeling fine overall. (non-safety-net)
Mental health issues	Just the chance of having cancer has me not sleeping well and feeling down. (safety-net)
	Suggestion to cease illicit chemical use. Current use replaces lonliness and lack of companionship. Relationship self esteem suggestions. (safety-net)
	I have my good days and bad days regarding depression and anxiety, but overall I'm stable. Anxiety is more pervasive and is unsurprisingly associated with stress. (non-safety-net)
Social issues	I continue to be about as isolated (safety-net)
	I've been working 60–70 hour weeks for the last couple of months--a regular job and a side gig. (non-safety-net)
	I have not been back to work. (non-safety-net)

Table 4.

Visit Goal Examples.

Example goal(s)	Goal(s) addressed in associated visit note
lymphedema - arm and legs/ core strength excessive sleep attacks - probably medicine related (safety-net)	All goal(s) addressed
1. Right shoulder pain 2. Apparent weakness in right leg (gluts, quadricep) that limits the distance I can walk (especially uphill). 3. I would like to get a flu shot. (non-safety-net)	All goal(s) addressed
follow up on: heart monitor, MRI, ankle x-ray, possibly rheum visit. (non-safety-net)	Some goal(s) addressed
1.I would like to focus on the following issues and concerns: overall medical issues mentioned on the above question and to some extent on my back pain. I do have a follow up with my surgeon in February. 2.I would be also discussing some social concerns or needs that might help me ease in my living standards in relation to my health issues. (safety-net)	Some goal(s) addressed
is my hip Pain normal. - Sometime back XX prescribed Metforman. I have run out of tablets and wondered should I continue to take it? (safety-net)	Goal(s) not addressed
Dizziness (non-safety-net)	Goal(s) not addressed

Of those who both completed a pre-visit questionnaire and attended their appointment (n = 56 safety-net, n = 97 non-safety-net), the majority of patient responses were added by the provider to the visit note (70% at safety-net clinic, 62% at the non-safety-net clinic) with no statically significant difference between the two clinics (p > 0.05). Patients at the safety-net clinic were less likely to have all of their goals addressed within the PCP documentation, compared to the non-safety-net clinic: 68% safety-net clinic, 84% non-safety net, a statistically significant difference (p < 0.01; Table 2).

Discussion

Electronic health portals are increasingly utilized for communication between patients and care teams, including early trials of co-generation of visit notes.[6,25] In this study comparing use of a novel pre-visit questionnaire asking patients to write an interim history and their goals for the upcoming visit, we found that questionnaires returned in a safety-net clinic and a non-safety net clinic had similar word counts, number of clinical history elements, and number of goals for the clinic visit, and that insurance type was not associated with whether or not a patient returned the questionnaire. However, use of the pre-visit questionnaire was very low at the safety-net clinic. Patients in the safety-net clinic also had fewer of their stated goals addressed in the provider's documentation of the visit, compared to the non-safety-net clinic; there were slightly more goals on average at the safety-net clinic compared to the non-safety-net which may have made it more difficult for all of the goals to be addressed.

Patients crafting their stories in their own words could enable them to feel “heard” by the provider, leave a permanent record of their story in the chart as they perceive it, and could improve clinic note accuracy. For patients who are disenfranchised from doctors and the medical system, this could be particularly empowering. More work will need to be done to determine if this tool improves trust with providers, or if non-digital engagement methods are more effective. Providers may also benefit from the pre-visit information for agenda setting and potentially more streamlined history taking and efficient visits. Our chart review of historical elements in the patient interval histories found that about half of patients submitted one or more historical elements, thus future revision of the questionnaire may be needed to gather more complete historical data. With recent changes to Medicare documentation requirements for billing to reflect medical complexity and time, counting elements of history are less important for billing, but patients documenting them could still save provider time and improve documentation quality and related provider burnout.

The COVID-19 pandemic required social distancing for patients with co-morbidities and moved telehealth and remote care via technology to the forefront, with a resulting gap in access for patients from vulnerable populations. Studies suggest that underserved populations are highly motivated to adopt new technology, but that access alone is not sufficient and tailoring technologies for users is needed. Prior work suggests that accessing health information is more likely if patients from vulnerable populations own smartphones, and that language influences preferences related to accessing electronic data. As technology becomes an increasingly important for patient engagement in health care due to the 21st Century Cares Act and the pandemic initiated telehealth options, policies will need to specifically work with vulnerable populations and address barriers to benefiting from these health care advances.

Provider access to patient responses also required technical skills in order to find the questionnaire and add them to the visit note; additional work is needed on the part of the electronic medical record system to improve providers’ ability to merge patient data into clinic notes to ensure ease of use for providers, and to create a work flow that routes the data to providers even if the patient misses the visit. Designing an interface that improves the visibility of the patient responses and ease of adding to the note may require physician training and improved user centered design. This study examined utilization at two different clinics, however, these clinics are associated with a single academic system meaning that our participants represent a small subset of patients and may not be widely generalizable. Similarly, the participant sample may not be representative of all respondents in the larger study. We did not notify patients about the study ahead of time; we simply activated it in the electronic portal and chart; this required patients to have numerous technical skills to participate, including being signed up for the portal, accessing the link from email, and knowing their password to login. We do not have information about patients who did not respond to the questionnaire. The pre-visit questionnaire was also only available in English; future electronic tools should include other languages and advances such as dictation options on smart phones to increase engagement for all populations. The patients included in this study had at least three visits to the healthcare system in the last year, and were greater utilizers of clinic services; however, even within this more engaged cohort, use of this novel communication tool was limited. Further research involving a broader spectrum of patients in varied clinical settings is needed in order to fully understand the optimal roles for co-generation of notes.

In conclusion, this novel questionnaire was utilized less by patients at the safety-net than the non-safety-net clinic. Respondents at the two clinics were more similar than different, with White patients more likely to use the tool. Developing electronic patient engagement tools that are accessible to non-English speaking patients, and investing in human resources, such as patient care partners or cultural and/or linguistic peer navigators, as well as improving provider training and access to patient responses like these, are opportunities that should be studied. As health care increasingly depends upon remote access to clinical teams and health care information via electronic portals, it is imperative that we invest in mitigating digital disparities.