Self-efficacy and confidence of medical students with prior scribing experience: A mixed methods study.

PURPOSE: Medical scribing is an increasingly common way for pre-medical students to gain clinical experience. Scribes are a valuable part of the healthcare team and have high rates of matriculation into health professional programs. Little is known about the effects of scribing on the success of the student. This manuscript aims to determine the effect of scribing experience on clinical self-efficacy during medical school. PARTICIPANTS AND METHODS: Perceived clinical self-efficacy was evaluated with validated survey questions using a 5-point Likert-type scale as well as free text responses. The survey was completed by 175 medical students at the Frank H. Netter, MD School of Medicine. Statistical analysis was conducted using SPSS. As part of the mixed methods study, free text responses were analyzed using thematic analysis.
RESULTS: Quantitative results showed no statistical difference in perceived clinical self-efficacy between medical students with scribing experience and those without. Analysis of free text responses showed that medical students believed their scribing experience improved comfort in the clinical setting and increased familiarity with medical terminology. DISCUSSION AND
CONCLUSIONS: Medical students with scribing experience did not demonstrate greater clinical self-efficacy than their peers without scribing experience. However, medical students with scribing experience have a perceived value of their pre-medical scribing experience on their success in medical school.

Introduction

In order to improve their applications, many premedical students undertake roles to increase their awareness of careers in medicine. Two such roles are medical scribing and shadowing. A medical scribe is a paid member of the healthcare team who performs documentation for healthcare providers with the goal of increasing efficiency [1]. Pre-medical shadowing is an unpaid opportunity that offers many benefits, such as increasing interest in medicine, seeing the field of medicine accurately, and improving confidence in clinical skills [2-6]. Pre-medical experiences have the greatest benefit when students are active participants. In a published personal reflection article, a former scribe who is now a physician stated that scribes are seen as part of the medical team as opposed to observers. One physician stated that her experience as a medical scribe improved her performance in medical school by becoming fluent in medical terminology, developing clinical etiquette, as well as having context for medical concepts taught in class. Through scribing, she gained a more accurate view of a career in medicine [7].

Many pre-medical students state that they have become a scribe to support their chances of matriculation at a medical school [3,6]. Admission committees desire to select applicants with high intellect and academic achievements [7,8]. However, pre-medical academic performance is not a good predictor of the quality of physician’s clinical or professional skills [9-12]. In one study, technician level clinical experience prior to medical school was associated with a poorer outcome on medical school standardized examination [13]. It is hard to define what will make a successful physician, so many schools are turning towards a holistic approach [11]. Pre-medical students will orient their experiences to be favorable to the medical school admission committees [5,14-16]. As such, it is necessary for pre-medical students to have access to accurate information regarding which types of experiences will benefit them.

Medical scribing companies appeal to pre-medical students by advertising the value of the experience [5]. Despite scribing being common among pre-medical students, there is surprisingly little literature concerning the positive or negative effects on student performance once in medical school.

While medical scribes improve the experience and efficiency of the physician-patient encounter [3,5,17] it is important to also define the benefits to the scribes themselves. To start answering this question, this study seeks to determine if medical scribing prior to matriculation into medical school improves clinical self-confidence of medical students.

Methods

Design

We conducted a cross-sectional survey of current medical students at the Frank H. Netter, MD School of Medicine to evaluate the scribing experience and perceived clinical self efficacy.

Survey development

Questions from two validated surveys were chosen by the authors based on their perceived relevance to medical student self-efficacy [18,19]. Each survey tool was validated on a population of students as part of their development. Permission was obtained for use of the survey questions as part of a larger questionnaire.

The demographic questions (Appendix Table 1) of the survey assessed information about medical students including matriculation year and military status. The intent was to stratify the data based on level of education rather than age, so age was not included in demographic questions. Military status was assessed because the school contained both civilian and military students. The demographic questions also assessed students’ scribing experience including location, duration, and administrative experience. Free text questions and a comments section allowed students to share their perception of how medical scribing influenced their medical school performance as well as a comments section. The clinical confidence questions (Appendix Table 2) assessed the students’ confidence (Cronbach’s alpha = 0.87) and self-efficacy (alpha = 0.94) on a 5-point Likert-type scale. Questions were omitted only if they were not relevant to the surveyed population.

Table 1.

Summary of Respondent Demographics

CHARACTERISTIC	FREQUENCY	PERCENTAGE
GRADUATION YEAR
2020 (FOURTH-YEAR)	36	20.6
2021 (THIRD-YEAR)	36	20.6
2022 (SECOND-YEAR)	41	23.4
2023 (FIRST-YEAR)	62	35.4
SCRIBE EXPERIENCE
YES	58	33.14
NO	117	66.9
SETTING OF SCRIBE EXPERIENCE
EMERGENCY DEPARTMENT	26	44.8
PRIMARY CARE OFFICE	14	24.1
ORTHOPEDIC OFFICE	2	3.4
URGENT CARE	2	3.4
NEUROSURGERY OFFICE	1	1.7
ONCOLOGY	2	3.4
OPHTHALMOLOGY OFFICE	4	6.9
DERMATOLOGY	1	1.7
MOBILE OUTREACH CLINIC	1	1.7
UROLOGY	1	1.7
CARDIOTHORACIC SURGERY	1	1.7
SETTING OF SCRIBE EXPERIENCE
URBAN	18	31.0
OUTER URBAN	33	56.9
RURAL	7	12.1
ADMINISTRATIVE EXPERIENCE
CHIEF SCRIBE	16	27.6
QUALITY ASSURANCE SPECIALIST	4	6.9
NEW SCRIBE TRAINER	23	39.7
NUMBER OF YEARS SCRIBING
0–1 YEARS	29	50.0
1–2 YEARS	21	36.2
2–3 YEARS	6	10.3
MORE THAN 3 YEARS	2	3.4
NUMBER OF HOURS WORKED PER WEEK
LESS THAN 8	2	3.4
8–16 HOURS	8	13.8
16–32 HOURS	23	39.7
32–40 HOURS	17	13.8
MORE THAN 40 HOURS	8	13.8
MILITARY STATUS
PRIOR MILITARY SERVICE	5	2.9
CURRENT MILITARY SERVICE	5	2.9
NO PRIOR OR CURRENT MILITARY SERVICE	165	94.3

Table 2.

Qualitative Analysis of Free Text Responses

THEME	NUMBER (PERCENTAGE)	EXAMPLE COMMENT
TERMINOLOGY	45 (75)	‘[S]cribing made me very comfortable with writing and understanding medical documentation […] very early on in medical school. I think it definitely gave me an advantage.’
MEDICAL DOCUMENTATION	45 (75)	‘Yes, it has helped me learn medical documentation and helped me feel more comfortable using EMRs’
PHYSICAL EXAM SKILLS	19 (31.7)	‘I heard about physical exam findings such as Murphy’s sign, Tinel’s sign, murmur grading, Cranial Nerve exams, and more.’
COMMUNICATION SKILLS	16 (26.7)	‘It exposed me to […] the different communications skills of a healthcare team.’
EXPOSURE TO MEDICINE IN GENERAL / UNDERSTANDING OF HOW MEDICINE WORKS	14 (23.3)	‘Scribing gave me a better sense of what happens to a patient as they move through the hospital system. Scribing allowed me to see the order or provider contacts the person might make (EMS, intake nurse, medic, nurse, PA/dr. etc.) and how long these processes take, which provider takes care of what responsibilities.’
ORAL PRESENTATION	14 (23.3)	‘Was much more proficient at oral presentation skills.’
PROCEDURAL SKILLS	8 (13.3)	‘Some procedural skills in M3 and M4 seemed to be easier to pick up since I had already been seen some multiple times.’
COMFORT WITH PATIENT INTERACTION	7 (11.6)	‘I feel less shy around patients.’‘It has […] helped mold my bed side manner and clinical approach to patients.’
BILLING PROCEDURES	5 (8.3)	‘I also saw ICD-10 codes and saw names for some of the diseases I would be encountering in patients.’
MEDICATION NAMES	2 (3.3)	‘I had some familiarity with commonly prescribed medications in Primary Care and their use (for example I could have told you that Lisinopril, Amlodipine, HCTZ were used for hypertension before stepping foot in a medical school).’
DEVELOPING A DIFFERENTIAL/ CLINICAL REASONING	2 (3.3)	‘The experience of seeing many sick patients helps develop a gut feeling for when you need to worry about a patient.’
CHANGED CAREER PLAN	1 (1.7)	‘It exposed me to medicine and opened the door to something I never imagined doing or what it was like.’

Participants

The survey link was distributed to all current medical students at Frank H. Netter, MD School of Medicine in the Spring of 2020 via student emails and the student Facebook group. Of 360 students, 207 (75%) accessed the survey and 175 (49%) completed the survey. Of the students who accessed the survey, 32 students (15%) did not complete the survey. No exclusion criteria were put in place because the survey was only accessible to current students, and all were eligible.

Survey administration

The method of completing the survey was via Qualtrics using an internet link. The survey link was sent out via email and Facebook group twice over a two-month period. One week after the last survey link was sent, the survey was closed, and data collection discontinued. The survey can only be accessed and completed once per student. A second survey followed the first which allowed for anonymous collection of participant emails, which was necessary for compensation. The informed consent statement was depicted on the first page of the survey, with the statement that stated clicking ‘continue’ denoted consent. Informed consent, participant compensation, survey content, and distribution methods were approved by the Quinnipiac University IRB.

Statistical analysis

Descriptive statistics included frequencies with percentages for ordinal variables and means with standard deviations for ordinal variables. One-way ANOVAs were used to compare demographic factors on confidence and self-efficacy. For factors measured at an ordinal level, a test for linear trend was also performed. Pearson correlation was used to assess the relationship between ordinal variables. Analyses were conducted in SPSS v26 and statistical significance was set at alpha of 0.05.

Responses to free text questions were reviewed using qualitative methodology via thematic analysis. Repeated themes of the free text were identified. Comments were then organized by theme. The number of comments in each theme were quantified and reported.

Results

Demographic results

Table 1 Shows the demographic characteristics of the n = 175 respondents who completed the survey.

Most respondents were first-years (35%) followed by second-years (24%), and lastly fourth-years (21%) and third-years (21%). Sixty-seven percent of students who responded to the survey did not have prior experience as a scribe. The majority of the 33% of the respondents with prior medical scribe experience did so in an emergency department (45%) followed by primary care office (24%) and then ophthalmology office (7%). Most scribes worked in an outer urban setting (57%), followed by an urban setting (31%) with rural settings being the least common (12%). Forty-three (74%) respondents reported administrative experience as a scribe; the most common being a new scribe trainer (40%), followed by chief scribe (28%) and quality assurance specialist (7%). Most scribes had scribed for less than 1 year (50%), which is consistent with the previously reported description of a medical scribing position as a stepping stone to other careers [15]. Only two (3%) respondents had worked as a scribe for more than 3 years. Most of the scribes worked between 16 and 32 hours per week (40%), while some scribes worked over 40 hours per week (14%) and two worked less than 8 hours per week (3%). Most scribes had no prior military experience (94%), while 5 respondents had prior military service, and 5 are currently serving in the armed forces (3% each).

Clinical self efficacy & confidence results

Figure 1 shows scores of overall clinical self-efficacy and level of confidence by scribe experience. While the average scores were slightly different for students without experience, the difference was not significant for self-efficacy [Yes: Mean = 3.3 (SD = 0.8), No: Mean = 3.4 (SD = 0.6), p = 0.22] or confidence [Yes: Mean = 3.5 (SD = 0.9), No: Mean = 3.7 (SD = 0.8), p = 0.10].

Figure 1.

Mean clinical self-efficacy scores of scribes vs. non-scribes.

Having an advanced scribe role, setting of scribing, number of years worked as a scribe, and hours worked per week as a scribe were all not statistically significant for self-efficacy.

The two variable years worked and hours per week worked were combined to create a new variable called total hours worked as a scribe. The mean number of hours total worked was 1169 (SD = 632) and ranged from 208 to 3120 hours. Figure 2 shows the scatterplot between total hours worked with self-efficacy and confidence. There was a positive relationship for both variables but were not significant and had low r values (self-efficacy: r = 0.26, p = 0.055, confidence r = 0.25, p = 0.062).

Figure 2.

Scatterplot with regression lines between total hours worked with self-efficacy and confidence.

Qualitative results

Table 2 Shows the rate of themes present in the free text responses of the n = 60 respondents.

Discussion

Medical school admission committees often desire applicants that have some sort of clinical experience in the medical field. Medical scribing is an increasingly common opportunity for clinical experience. By using survey methodology and qualitative analysis, this study starts to fill a gap of information in the literature for pre-medical students and their advisors on the utility of scribing for success in medical school.

The survey portion of this study showed that students who were medical scribes prior to matriculation to medical school did not have any statistically significant differences in their clinical self-efficacy when compared to students who were not scribes. However, a qualitative analysis of free text responses revealed that students who were medical scribes prior to matriculation to medical school felt that their experience was extremely influential on their performance in medical school.

The discrepancy between the validated survey results and the qualitative analysis leaves at least three possibilities. The first is that the validated surveys did not adequately capture the unique qualities of scribing that lead to improved self-efficacy when compared to peers. Secondly, the students with scribe experience may have their experience was more valuable than it actually was. Thirdly, it is possible that non-scribing clinical experience and medical scribing experience were equally as beneficial to matriculated medical students’ clinical self-efficacy, so the differences were not apparent in a survey that only evaluated scribing experience.

To investigate this discrepancy, it would be useful to determine if students who had pre-medical clinical experiences that were different from scribing felt that it was beneficial to them for similar reasons as the students who had scribing experience.

There are several limitations to this study. First, all survey-based studies are subject to recall bias. The survey had a small sample size and should be expanded to include many medical schools across the country. Of those who responded, over half were pre-clerkship students. Pre-clerkship students at Frank H. Netter, MD School of Medicine participate in a once-per-week primary care patient care experience, but still have limited clinical medicine experience. While the survey response rate of 49% is high, it leaves 51% of the students at this institution as not being part of the sample. Students were compensated $5 Amazon gift cards for participation in the survey. While this amount was approved by the IRB and was given to all participants, it does introduce a potential bias. The questions from the survey in this study came from two individually validated surveys, but the combined survey was not validated. Further studies should pilot and validate the survey used in its final form.

Next, the inability of the surveys to distinguish between medical scribing experience and non-scribing clinical experience leaves the question of whether these two types of opportunities reap any difference in benefit to the student. It would be interesting and useful to research if observational experiences (shadowing, medical scribing) are as beneficial as hands-on experiences (technician jobs, medical assistant). Additionally, running focus groups on students with different types of pre-matriculation experience may illuminate perceived differences. Overall, further research needs to be done on the variety of pre-medical opportunities and how these experiences influence a student’s success in medical school.

#	QUESTION	RESPONSE OPTIONS
1	When will you be graduating medical school?	2020 2021 2022 2023
2	Did you have experience working as a medical scribe prior to entering medical school?	Yes No
3	Setting of prior scribe experience capacity:	Emergency Department Primary Care Office Other [Free text]
4	Administrative experience	Chief Scribe/ Manager Quality Assurance Specialist New Scribe Trainer Other [Free text]
5	Number of years spent working as a medical scribe	0–1 1–2 2–3 more than 3 years [Free text]
6	Approximate number of hours per week worked as a medical scribe	Less than 8 hours per week 8–16 hours per week 16–32 hours per week 32–40 hours per week more than 40 hours per week
7	Did scribing experience influence your clinical experience in medical school? Please add any comments to the text box. (Medical documentation? Terminology? Communication skills? Procedural skills? Physical exam skills? Oral presentation skills?	Free text
8	Describe the setting of your scribe experience	Urban Outer Urban Rural
9	Describe your military status	Prior military service current military member no prior or current military experience
10	Comments?	Free text

#	QUESTION
1	Apply knowledge of normal function to each of the major organ systems?
2	Effectively manage the uncertainty associated with patient care, such as when the patient has multiple treatment options, each with its own risks and benefits?
3	Apply knowledge of epidemiology of common diseases, such as heart disease, to reduce disease incidence?
4	Use effective listening skills when interacting with a patient?
5	Demonstrate caring when counseling a patient?
6	Accurately gather essential information from a patient?
7	Perform a thorough physical exam?
8	Develop an appropriate differential diagnosis?
9	Generate a patient-specific treatment plan?
10	Use information technology to support patient-care decisions?
11	Work effectively with other healthcare professionals to provide high-quality patient care?
12	Improve clinical practice using a systematic approach?
13	Evaluate evidence from scientific studies relevant to your patients’ health problems?
14	Stay abreast of relevant scientific advances by reading peer-reviewed medical journals?
15	Demonstrate sensitivity to patients’ cultural differences?
16	Balance professional responsibilities with personal responsibilities?
17	Discuss methods of controlling healthcare costs?
18	Practice cost-effective healthcare delivery that does not compromise quality of care?
19	Deal with ethical dilemmas
20	See patients independently
21	Relate to a diverse patient population
22	Be a self-reflective practitioner
23	Explain medical information to patients
24	Contribute to patient care
25	Describe the natural history of illness over time