Factors Associated with Academic Performance in Physician Assistant Graduate Programs and National Certification Examination Scores. A Literature Review.

PURPOSE: A physician assistant (PA) is a state-licensed, nationally certified healthcare professional who practices medicine on healthcare teams with physicians and other providers. PAs practice medicine across the US (all 50 states, the District of Columbia, and the US territories). In recent years, the demand for clinicians has increased dramatically which has led to an increase in the number of practicing PAs. To meet this growing demand for healthcare providers, identifying applicants capable of overcoming the challenges associated with the PA educational track in addition to the corresponding clinical training is crucial.
METHOD: In this paper, we reviewed the literature and discuss preadmission factors and their relationship toward completion of PA graduate programs and successfully passing the national certification examination (PANCE).
RESULTS: Previous studies indicated a weak positive association between verbal GRE scores and success on the PANCE. Moreover, undergraduate GPA, and taking a variety of undergraduate science prerequisites correlates with passing the PANCE. DISCUSSION: Investigations of success correlates of other professional programs indicated that psychological factors may have potential for use in predicting whether an applicant would be successful in PA school. These include tests for emotional intelligence and particular personality characteristics.

Introduction

The shortage in healthcare professionals has been a problem for years, and will intensify with the anticipated demand for medical services by the baby boomer generation.[1] The predicted number of unfilled positions of primary care physicians by 2025 is 124,000, and although enrollment in medical schools has recently risen by 30%, this will not sufficiently offset the tremendous shortage in healthcare providers.[1,2] To compensate for this overwhelming need, there has been a trend for increased reliance upon other primary care providers which includes physician assistants (PAs). There are currently over 100,000 actively working PAs in the US, with a projected 37% increase in the rate of employment through 2025 according to the Bureau of Labor Statistics.[3] With clinician demand growing, there is a greater burden placed on PA academic programs to identify individuals capable of the rigor associated with this track of graduate education as well as the corresponding clinical training. Therefore, identifying student characteristics associated with success or failure in PA school may allow these programs to better identify candidates who will successfully complete the associated studies and become practicing clinicians.

In recent years, PA programs have experienced a dramatic increase in the number of applications likely due to the high level of career satisfaction among current PAs, the lateral mobility associated with this career path, as well as the desirable salary.[4] Despite the abundance of applicants, only a limited number of prospective students are chosen for program entrance.[4] Although a rigorous screening process leads to the selection of seemingly qualified students, attrition still occurs among enrolled PA students (5.2%) counteracting efforts to increase the number of clinicians and reducing the potential revenue for the academic institutions.[1,5] Moreover, selecting students who are more likely to complete PA school would be desirable these programs as retention strengthens the economic stability of the academic institutions. In addition to completing the academic and clinical curricula before becoming a practicing PA, students must take and pass the Physician Assistant National Certifying Examination (PANCE). PANCE pass rates are evaluated by accreditation bodies, and can influence the number and quality of future applicants.[1] Therefore, academic programs would benefit from determining factors that predict success, decrease attrition, and increase PANCE performance.

Currently, standardized selection criteria for admission into PA programs have not been established. This may be due to the diversity of missions between different academic institutions as well as the lack of research to identify valid, generalizable selection criteria.[4] Existing research on predictors of success is variable and at times contradictory. PA programs consider various pre-requisites for admission including undergraduate science GPA minimum, volunteer hours, shadowing hours, direct patient care, and specific coursework, in addition to many other factors.[4] The Physician Assistant Education Association publishes an annual program report containing an abundance of descriptive data generated from a survey completed by PA programs [210 participants in 2016 [5]]. While this report is valuable for showing trends in PA education, determining which factors are associated with academic success not possible from the summary data provided. For example, this report indicated that the average science GPA for incoming students into PA programs was 3.50.[5] However, this report did not provide an analysis of whether pre-admission GPA was associated with completion of the PA program.5 Controlled studies and analyses of cohorts in which researchers determine whether such factors are associated with performance in PA academic programs are scarce in the literature.

PA graduate programs typically involve a didactic section followed by a clinical portion where candidates actively learn from licensed clinical preceptors. Notably, a significant hurdle for PA program administrators is to secure adequate numbers of sufficient clinical placement sites. While we acknowledge that this represents a major limiting factor restricting the number of enrolling PA students, particular student variables have been associated with success in PA education. Identifying these factors can help ensure that academic programs select students who have the highest likelihood of success which would limit attrition. These variables reviewed in this article include: entrance examination scores, undergraduate GPA/foundational coursework, personality testing, emotional intelligence, and coping methods in addition to other demographic factors.[4] Analyzing these factors may lead to the establishment of a screening standard to increase retention, completion, and PANCE pass rates for PA students. A screening standard will help to increase output of competent clinicians by making sure programs accept students who will be able to successfully complete a program. Subsequently, this will help to aid in supplying health-care professionals for the impending shortage.

Demographics

Generally, reputable academic institutions do not discriminate based on demographic factors such as race, religion, and gender. In fact, Title VII of the United States federal law prohibits such practices. Therefore, any association between these characteristics and PANCE success would and should have little bearing on admission decisions. We suspect, because of these ethical principles and federal laws, studies on the influence of demographic factors on success in PA school are limited. Nevertheless, some have investigated whether demographic factors such as gender and age are associated with successful completion of a PA program and passing the PANCE. According to Asprey et al., age is a negative predictor of PANCE success as older test takers scored significantly lower on this exam than younger examinees, especially among males.[6] Moreover, this study reported that the likelihood of PANCE failure significantly increased with age. In addition to exploring whether or not age was associated with failure or success on the PANCE, Asprey et al. also evaluated the influence of gender.[6] This study showed that PANCE scores of females were significantly higher than those of male test takers.[6] Contrary to the results of Asprey et al., Higgins et al. and Oakes and colleagues reported that neither age nor gender influenced PANCE performance.[6-8]

Entrance exams

The Graduate Record Examination (GRE) is required by over half (~51%) of PA programs.[5] However, investigations into the usefulness of this exam for predicting student success on the PANCE have produced mixed results. A study performed by Butina et. al. reported the verbal section of the GRE is weakly associated with success on the PANCE, while the mathematics section of the GRE has shown no meaningful direct effect on PANCE performance.[9] Because of this weak association, the broad applicability of the GRE to predict success is likely unreliable. In support of this conclusion, a study conducted by Higgins et. al. reported that both verbal and mathematical GRE scores were significant predictors in only two of the six PA programs evaluated.[7] Therefore, GRE scores are likely inadequate or at least inconsistent predictive measures.[7]

Medical schools and other similar graduate programs require applicants to take the Medical College Admission Test (MCAT). This exam is intriguing because it analyzes an applicanťs ability to think critically, problem solve, and demonstrate knowledge in advanced science. For years, this exam has been used to provide medical schools assessment information on applicants, and multiple PA programs have begun accepting the MCAT in combination with or as an alternative to the GRE [15% of PA programs [5]]. Because of this, investigating the likelihood of the MCAT for predicting success in PA programs may be of interest.

Undergraduate and graduate factors

In 1910, the Flexner Report compelled medical schools in the US to implement higher standards for both admission as well as graduation. Moreover, this report transformed education in medicine within the US to a scientifically-based model for medical training. This report was utilized to expose the insufficient quality of medical education and the aftermath of this study led to a substantial increase in the quality of the clinician and an acknowledgement of the importance of scientific knowledge in medical education.[10] One of the many changes that Abraham Flexner recommended was that applicant selection be dependent on competent knowledge in biology, physics, and chemistry. Flexner asserted that any departure from this foundation would lead to a detriment in medical training.[11] Undergraduate GPA, GPA in science courses, and the repertoire of science courses taken have historically been used to evaluate incoming students for admission into PA programs.[6] Several reports have shown these criteria to be viable indicators of success and performance on the PANCE.[6] For example, Higgins et al. evaluated numerous parameters and found among these, undergraduate GPA were among the most reliable predictors for success on the PANCE.[7] Further support came from Andreef[1,2] who showed that higher admission GPA was positively associated with first-attempt PANCE scores.

Although many PA schools use undergraduate GPA and performance in science classes. as criteria for student selection, recent studies have called to question the reliability of these criteria to predict success. For instance, in 2007–2009 three graduating cohort classes were evaluated from the Wichita State University PA program in a retrospective study conducted by Brown et al.[4] This program accepted 42 students during this time frame, all of which had met prerequisite undergraduate GPA requirement of 3.0. Along with evaluating both the undergraduate and science GPA, the repertoire of science class prerequisites was also evaluated. These classes included general biology, general chemistry, human anatomy and physiology, and microbiology. Students whose data were unavailable from those prerequisite courses or those who withdrew from the program were not evaluated in this study. The results from the remaining 119 students did not reveal a correlation between undergraduate GPA and performance on the PANCE nor was science prerequisite GPA associated with the PANCE performance.[4] This study instead revealed a weakly positive association between taking science prerequisites and PANCE pass/fail rate.[4] However, a post-admission factor (GPA/performance during didactic phase) positively correlated with PANCE pass/fail rate.[4] Consistent with the findings of Brown et al., a study published in 2017 revealed further ambiguity for the reliability of undergraduate GPA to predict PANCE and PA school success.[9] Butina et al. showed that while the criteria required for admission into PA school provided some indication on PANCE performance, the most reliable predictor of success was performance on foundational coursework taken in the PA graduate program.[9]

Although admission criteria provide some indication of how a student might perform on. the PANCE, foundational coursework is seemingly a reliable early predictor of PANCE. performance.[9] Future studies should therefore analyze whether foundational coursework is associated with PANCE performance.[9] Although earning high grades during the foundational didactic phase of PA school correlates with PANCE success, this predictor is a ‘post-admission’ factor, and does not provide insight to PA school admission boards.[9] Other post-admission factors that have been shown to correlate with success include PACKRAT scores (the PAEA Physician Assistant Clinical Knowledge Rating and Assessment Tool)[13] and the composite 7-item End of Rotation score (these exams are validated, objective and pertain to relevant medical knowledge in seven core clinical areas)[14]. While these post-admission criteria seem to be predictive of success, Board Review courses do not seem to affect PANCE performance.[15] Moreover, Colletti et al. showed that shorter length PA programs produced similar PANCE scores than those programs that encompassed longer time periods.[16]

Emotional intelligence

Programs educating future healthcare professionals have increasingly emphasized emotional intelligence (EI) as an indicator of interpersonal communication skills and academic success. For instance, medical school students with higher EI scores performed better on continuous assessments and clinical examinations.[17,18] Other studies have explored correlations between EI and selection scores for admission (MCAT and interview) but found no significant association between these criteria.[18,19] Overall, EI testing is prominently used for measuring desired personal attributes (empathy, understanding, compassion) of medical school applicants, and may have potential as a selection criterion for admission into PA programs.[17,20] Although the model of education differs, accelerated graduate entry nursing programs have also associated higher levels of EI with increased levels of awareness and academic achievement.[21,22] Similar studies evaluating nursing students found EI to be negatively associated with perceived stress, and positively correlated with well-being, problem-focused coping, leadership skills, and perceived competency.[23,24] A significant predictive relationship was found between EI and retention of students after controlling for prior academic achievement, age, and gender.[22] Therefore, utilizing results from other healthcare programs may provide insight toward identifying selection criteria associated with success in PA programs. A multi-institutional and multi-professional study did not show significant differences in EI in students from medical school, dental school, or graduate level nursing programs suggesting that this criterion may be universal among successful graduate students in healthcare fields.[25] Given the research outcomes from other healthcare professional schools, PA programs may benefit from EI screening.

Personality testing

Proficient cognitive ability is an essential factor for success in a medical program, as individuals must be able to comprehend and process a vast amount of material during matriculation.[26] While research conducted on the use of personality assessments in PA programs is unavailable, studies have evaluated the use of such tests to identify candidates for medical school. Medical schools evaluate personality traits via personal statements, interviews, letters of recommendation, and in some cases psychological testing.[27] Psychological personality tests have shown to be a beneficial supplement in the candidate selection process.[26-29] Therefore, consideration of these factors prior to entrance into a medical program may contribute to the identification of successful candidates.[27]

The most widely used personality system among psychologists is the Five Factor Model where traits have been divided into five major categories: openness, conscientiousness, extraversion, agreeableness, and neuroticism.[29] Of these characteristics, conscientiousness is associated with successful performance in both academic and work environments.[26-28] A study conducted by Lievens et al. considered how personality factors would longitudinally contribute to academic performance in medical school. Conscientiousness was found to be moderately predictive of success in the early years and decidedly more predictive for later years.[26] Similarly, an additional study found that conscientiousness scores were higher for medical students who had successfully passed three preclinical year tests when compared to those students who had not passed all three tests.[26] These findings suggest that higher conscientious scores on a personality test may be beneficial in predicting potential medical students’ success in a program. Because individuals with conscientiousness have been shown to persevere, and utilize strategies to overcome stress, they may be more likely to succeed in many settings.[26,27,30] PA schools should evaluate whether conscientiousness could be useful for predicting success, and whether or not personality tests should be applied to future cohorts of prospective students.

Extraversion is another personality trait that may be considered.[30] Extraversion differs from conscientiousness in that this trait appears to have a curvilinear relationship with success throughout a medical program. For instance, in the preclinical years of medical school, individuals with higher extraversion scores tended to have lower GPAs.[29] Students who scored high on gregariousness and excitement seeking and low on the conscientiousness factors of achievement striving and self-discipline were more likely to drop out of the program.[29] However, in the later clinical years of medical school, when internships are of great importance, high extraversion was found to be a significant predictor of success.[26] An explanation of these results may be that students who are highly extraverted may be distracted by their desire to participate in social endeavors that take away from study time, which would negatively affect their grades in the didactic years. On the other hand, the high levels of sociability and positivity would be advantageous during the clinical phase of school. We propose that students with high conscientiousness and extraversion scores would find the most success in PA programs. However, future research is required to test this hypothesis.

Coping methods

Several studies have been conducted that examine the mental demands medical school has on students, and the coping mechanisms that could be incorporated to deal with the associated challenges. For instance, a six-year longitudinal, nationwide and comparative study was conducted to evaluate life satisfaction and resilience of students in medical school.[31] This study showed that life satisfaction of medical students was comparable to students in analogous academically rigorous programs, but decreased through the duration of medical school.[31] Particularly, the level of life satisfaction in the final year of medical school was low compared to students with equal academic rigor. Although medical school is much longer than PA school, PA students are required to learn a condensed amount of material in a short period of time. With this, we predict that life satisfaction and stress levels of PA students may be comparable to medical students. Therefore, encouraging PA students to spend adequate time on their social and personal lives as well as stress management may increase life satisfaction which could potentially impact success in PA school.

A study by Yusoff et al. investigated the effectiveness of stress-management programs on the coping ability of medical students.[32] This systematic review indicated that students who participated in stress-management programs were better able to cope with the stress and deleterious effects associated with medical school.[32] This study showed that students who participated in stressmanagement programs exhibited a better functioning immune system, lower levels of anxiety and depression, higher empathy and spirituality, better coping skills, better knowledge of the effects of stress, heightened knowledge of alternative therapies for future referrals, and the ability to solve role conflicts.[32] Therefore, we predict that PA students who possess these qualities have a greater possibility of being successful on exams and enjoying medical school to a greater extent.

The rigors of graduate-level medical education can lead to emotional exhaustion and cognitive weariness, a phenomenon referred to as “burnout.” A study by Fares et al. found that burnout manifested very early in preclinical medical students as a result of work overload and the academic pressure to learn a great deal in a limited time frame.[33] Burnout and high stress levels are closely associated with decreased life satisfaction, serious thoughts of dropping out, and suicidal ideations.[33] These feelings are associated with poor performance on tests and reduced commitment which could exacerbate stress levels.[33] If coping strategies to reduce burnout were incorporated into PA programs, commitment levels and life satisfaction would likely increase, and therefore we predict student success would improve.

Discussion

Understanding association between PA student success and preadmission factors can help academic programs find individuals who will successfully complete PA school and will ultimately contribute toward alleviating the clinician shortage in America today.

The GRE is commonly required for PA school admission, although there have been few studies investigating whether or not high GRE scores are associated with PA student success. The results from available studies are inconsistent with one showing a weak association between verbal GRE scores and PANCE performance. Multiple larger longitudinal studies are needed in order to make a definitive conclusion about the effectiveness of the GRE. Some schools are accepting the MCAT for PA school admission although there is not enough evidence to make a conclusion on whether or not scores of this examination are predictive of PA school success. It may also be useful to evaluate other standardized testing such as the ACT or the SAT and their relationship with PA student success. Although, given the lack of an association between the GRE and PANCE performance, we predict that ACT and SAT scores will likely show a weak or no association with PA school success.[9]

Interestingly, research has shown that psychological factors may have potential for use in predicting success in PA school. EI in particular has been shown to predict how well a student can adapt to stress as well as their ability to think critically. Further research evaluating EI with respect to success in PA school may further the findings in these studies. Being able to effectively assess an applicanťs EI prior to admission could help with program retention rates and overall student success.

Personality factors can also give insight to how a potential applicant will handle the graduate school workload. Certain personality types have been shown to decrease fatigue among medical students. Specifically, conscientiousness has been shown to be moderately predictive of success in the early years and much more predictive of academic achievement in the later years of education. Having the ability to identify personality factors may provide insight to the admissions boards as to how a student will perform in PA school. Further studies involving preadmission personality tests and their connection with student success is warranted.