Content Evaluation of Integrated Plastic Surgery Residency Program Websites Regarding Sub-internships.

Sub-internships are an integral part of the residency application process for competitive specialties, including plastic and reconstructive surgery (PRS). This study evaluates the quality of online information regarding sub-internships offered by PRS programs.
METHODS: All the US integrated PRS residency programs were identified on Fellowship and Residency Electronic Interactive Database. Each program's website was assessed for the presence of 4 criteria: mention of a sub-internship offered, contact information of rotation coordinator, learning objectives, and a web page dedicated solely to the sub-internship. Each residency program was assigned a website sub-internship (WSI) score from 0 to 4 accordingly.
RESULTS: We identified 81 residency programs. Fifty-two residency program received a score of 0, and 6 programs received a score of 4. The mean WSI score was 1.0 for community-based programs, compared with 0.98 for university-based programs (P = 0.93). There was a correlation between WSI score and number of faculty (r = 0.52; P < 0.0000) and number of residents per year (r = 0.34; P = 0.002). In logistic regression model, number of faculty [odds ratio (OR), 1.18; P = 0.003], large program size (OR, 12.1; P = 0.009), and southeast location (OR, 30.3; P = 0.034) were found to be important predictors for PRS programs to mention at least one criterion.
CONCLUSIONS: A majority of the programs did not have any online information available on their websites regarding plastic surgery sub-internships. There is no difference between academic and community-based programs with regard to the quality of information. Programs with more trainees and faculty have higher odds of providing information about sub-internships offered at their institution.

INTRODUCTION

Plastic and reconstructive surgery (PRS) is one of the most competitive specialties for senior medical students to secure a residency. In the 2019 National Residency Match Program, 78 residency programs offered a total of 172 residency positions to 234 applicants, with an overall match rate of 73.5%.[1] Hence, residency applicants devote a significant amount of time and effort to put together a strong, competitive application for the residency match process.

Sub-internships (sub-Is) are usually 4-week rotations that fourth year medical students complete, and they have become an integral part of preparing medical students for the critical transition between medical school and residency training.[2] Over recent years, senior medical students applying into competitive specialties enroll in sub-Is at residency programs of their interest, in addition to their home institution. Under these circumstances, the purpose of completing sub-Is at other institutions goes beyond the original goal of sub-Is, which was to provide a strong foundation for real-life clinical performance before medical students graduate. Currently, sub-Is provide an opportunity for both residency programs and residency applicants to evaluate each other for an extended period of time on many intangible factors that could not otherwise be communicated. This includes the rotating student’s interpersonal and communication skills, the culture of the program, and the comradery between residents and attending physicians.[3] Although there are no studies in the literature that demonstrate the importance of completing away sub-Is in creating a competitive residency application, fourth year medical students interested in PRS are often advised to complete away sub-Is at more than one institution. In addition to the reasons stated above, the familiarity created through completing a sub-I between residency programs and permits applicants to separate themselves from the rest of the application pool. Moreover, although it is not guaranteed, students who successfully complete a sub-I often are offered an interview.

Previous studies in the literature have demonstrated that plastic surgery residency websites (PSRWs) are an important resource for prospective applicants at different points of their undergraduate medical training.[4] However, to the best of our knowledge, no study has evaluated residency program websites with respect to the information provided on sub-Is offered at their institution. The purpose of this study is to determine the availability and quality of online information regarding sub-Is on plastic surgery residency program websites.

METHODS

We identified all the integrated PRS residency programs in the United States on Fellowship and Residency Electronic Interactive Database (FREIDA). Using the Google search engine, the website for each PRS residency program was accessed. The search term always followed the same format: “X plastic and reconstructive surgery residency,” where “X” was the name of the program listed on FREIDA. All the residency program websites were evaluated by the author (F.D.), from October 31, 2019, to November 8, 2019. This study was exempt from Institutional Review Board approval because there were no human or animal subjects involved.

Given the paucity of data regarding evaluation of online content related to sub-Is, we used a grading scheme developed by Rai and Sabharwal[3] in their study evaluating the online content of orthopedic surgery residency program websites regarding sub-Is. Each website was assessed for the following 4 criteria: mention of a sub-I offered by that program, contact information of the sub-I coordinator, a list of learning objectives to be met by the rotating student during the sub-I, and the presence of a web page dedicated solely to PRS sub-I. Each residency program was given a website sub-I (WSI) score from 0 to 4 based on how many of the above criteria were met. The WSI score is only reflective of quality of online content provided on the residency program websites, not the quality of sub-I itself. Additional information obtained from each website included total number of residents over the years, average number of residents per year, and the number of full-time core faculty. Programs were also stratified based on program size, their geographic location (northeast, southeast, west, midwest, and southwest), and program type (academic-based or community-based). Academic-based and community-based categories were created based on the classification system used on FREIDA. The data were analyzed for possible relationships between the WSI score and program type, program location, number of full-time faculty, and number of residents.

Statistical analyses and figure productions were performed using STATA MP16 (StataCorp, College Station, Tex.). The unpaired t test was used to compare continuous variables with normal distribution and Wilcoxon rank sum test and Mann–Whitney U test were used for nonparametric continuous variables. χ2 test was used for comparing ≥2 categorical variables. Pearson correlation and spearman rank test were used to assess the correlation between 2 continuous variables. Because WSI scores did not follow a normal distribution, nonparametric statistical analyses were performed after assessing normality using Shapiro–Wilk test (P = 0.000). Multivariate logistic regression was used to analyze the outcome of mentioning at least one criterion of WSI score on the residency program website.[5] The predictor variables included in the model were program type, program location, number of full-time core faculty, and program size. Program size was defined based on the number of residents in the integrated program per year. Care was taken to ensure that residents in the independent tracks did not count toward the total number of residents in the integrated programs. Programs accepting 1 resident per year were considered “small” programs, programs accepting 2 residents per year were considered “medium” programs, and programs accepting ≥3 residents per year were considered “large” programs. The significance level was set at P = 0.05.

RESULTS

Univariate Analysis

We identified 81 integrated PRS residency programs in the United States, which is composed of 62 (76.5%) academic-based programs and 19 (23.5%) community-based programs. An estimated 52 (64.2%) residency program websites did not make any mention of a sub-I and received a score of 0; 2 (2.5%) programs received a score of 1; 9 (11.1%) received a score of 2; 12 (14.8%) received a score of 3; and 6 (7.4%) received a score of 4 (Fig. 1). For residency programs that received a WSI score of 0, we used Visiting Student Application Services (VSAS) to determine whether these programs did not offer a sub-I at all or whether they offered sub-Is but did not include any information on their websites. We found that 34 of 52 programs that received a WSI score of 0 indeed offered sub-Is by listing their rotation on VSAS but did not provide any information on their residency program website. Among the criteria analyzed for assigning a WSI score, mention of a sub-I offered and contact information of sub-I coordinator were most frequently encountered (Table 1).

Fig. 1.

Sub-internship score frequency distribution for WSI scores ranging from 0 to 4. The majority (64.2%) of programs received a WSI score of 0, and only 12% of programs received a WSI score of 4.

Table 1.

WSI Criteria Distribution among Programs

WSI Score	No. Programs (%)
Mention of a sub-internship offered	29 (35.8)
Contact information of the sub-internship coordinator	22 (27.2)
Learning objectives for medical students	9 (11.1)
Webpage dedicated solely to sub-internship information	20 (24.7)

The mean WSI score and standard deviation for community-based programs were 1.00 (1.49) in comparison to 0.98 (1.41) for academic-based programs (P = 0.93). Correlation analysis revealed a high degree of correlation between WSI score and number of full-time faculty at each residency program (r = 0.52; P = 0.0000) (Fig. 2). In addition, there was a moderate degree of correlation between the WSI score and the number of residents per year (r = 0.34; P = 0.002) (Fig. 3). We also performed a subgroup analysis comparing no information at all (WSI score = 0) versus some information (WSI score = 1–4) provided on PSRW regarding sub-I (Table 2). In this subgroup analysis, there was no statistically significant difference based on program type (P = 0.914) and geographic location (P = 0.382).

Fig. 2.

Scatter plot with trendline exploring the relationship between the WSI score and the number of full-time faculty. The correlation analysis demonstrated a high degree of correlation between these 2 variables (r = 0.52; P = 0.0000).

Fig. 3.

Scatter plot with trendline exploring the relationship between the WSI score and the number of residents per year. There was a moderate correlation between these 2 variables in correlation analysis (r = 0.34; P = 0.002).

Table 2.

WSI Score Outcomes Based on Various Factors

	No Information (WSI = 0)	Some Information (WSI = 1–4)	P
Type of program
University based	40 (64.5)	22 (35.5)
Community based	12 (63.2)	7 (36.8)	0.914
Size of the program
Small	21 (84)	4 (16)
Medium	21 (63.6)	12 (36.4)
Large	10 (43.5)	13 (56.5)	0.014
Geographic region
Northeast	16 (72.7)	6 (27.3)
Southeast	9 (47.4)	10 (52.6)
West	9 (69.2)	4 (30.8)
Midwest	13 (61.9)	8 (38.1)
Southwest	5 (83.3)	1 (16.7)	0.382

Multivariate Analysis

Multivariate logistic regression analysis was used to assess the outcome of including some information related to plastic surgery sub-I on PSRW, and the results are shown in Table 3. Large residency programs, program location in southeastern region of the United States, and increasing number of full-time faculty were associated with increased odds of including some information related to plastic surgery sub-I (P = 0.009, P = 0.034, P = 0.003, respectively).

Table 3.

Multivariate Logistic Regression Analyzing the Outcome of Mentioning ≥1 Criterion on Plastic Surgery Residency Program Website

	Odds Ratio (95% Confidence Interval)	P
Program type
Community	1 (reference)	—
Academic	0.72 (0.16–3.15)	0.661
Program size
Small	1 (reference)	—
Medium	4.88 (0.95–25.0)	0.057
Large	12.1 (1.89–78.1)	0.009
Program location
Southwest	1 (reference)	—
West	3.40 (0.17–66.9)	0.421
Northeast	3.41 (0.16–72.0)	0.430
Southeast	30.3 (1.29–708.5)	0.034
Midwest	9.85 (0.51–188.6)	0.129
Full-time faculty	1.18 (1.06–1.32)	0.003

DISCUSSION

Our study shows that the quality and availability of online information regarding sub-Is on PSRWs are variable. More than half of the PSRWs do not provide any information regarding offering plastic surgery sub-Is for prospective applicants. There is evident lack of online information for residency applicants to choose at which programs to complete their sub-Is. Notably, less than a quarter of residency programs provided detailed information about their sub-I by describing learning objectives to be achieved by rotating students.

Sub-Is are a crucial component of having a competitive residency application. At the end of their third year, medical students applying into PRS turn to the internet to obtain information regarding away rotations at other institutions. However, residency programs are not fully using this critical communication tool with prospective applicants. This should raise concerns for residency programs because it is plausible that residency programs with well-maintained and comprehensive websites are able to attract more competitive applicants. Moreover, it is also possible that this lack of uniformly accessible information can have negative impacts on the overall interest of medical students in the field of PRS. This study sheds light on the importance of the quality and availability of online information regarding plastic surgery sub-Is to guide residency programs in their efforts to attract and recruit top candidates who are the “right fit” for their program.

A notable finding in our study was that there were no statistically significant differences in mean and median WSI score between academic and community-based residency programs. This is in contrast to the findings of Rai and Sabharwal[3] in their study evaluating online content of orthopedic surgery residency websites regarding sub-Is. More specifically, they found that academic programs provided more information compared with community-based programs. As a general rule, community-based residency programs are not affiliated with a medical school or university. Hence, medical students are not typically exposed to them during their undergraduate medical education; therefore, they are less likely to complete a sub-I at such programs. To compensate for this factor, community-based programs may be attempting to provide information at a level comparable to academic programs. Another possible explanation for this lack of difference between mean WSI score between academic and community-based programs is that 17 of 19 community-based programs in plastic surgery are affiliated with a university as described on FREIDA. Hence, the academic component of these residency programs that oversights resident recruitment are handled by the affiliated university, and consequently, we did not see any significant differences based on program type.

Another important finding in our study was the presence of a high degree of correlation between the number of full-time core faculty and WSI score. This finding was further supported as the number of full-time faculty was found to be an important predictor in including some information related to sub-Is on PSRW independent of other predictors and confounding variables. This could mean that residency programs with more full-time faculty are potentially more established, and they are more likely able to accommodate medical students for a sub-I.

Similarly, in our correlation analysis, there was moderate degree of correlation between the number of residents per year and WSI score. The results of multivariate regression analysis were consistent with this finding, as larger residency programs were associated with increased odds of including at least one sub-I criteria examined on PSRW. Because residents are commonly the point person for medical students on sub-Is, it may be that larger residency programs are better equipped to accommodate medical students. Hence, they provide more information on their website regarding sub-Is.

Another interesting finding from multivariable logistic regression analysis was that residency programs located in southeastern United States had higher odds of including at least one criterion compared with programs in other regions. A possible explanation for this finding is that there are fewer historic residency training programs in southeastern United States relative to other parts of the country. Hence, residency programs in this region are including more information on their residency programs’ website to stand out to applicants and increase their publicity.

As sub-Is have become an integral part of the residency application process for senior medical students, residency programs and Association of American Medical College have put forth efforts to streamline the completion of a sub-I at another institution. Therefore, the majority of residency programs use the VSAS application portal to communicate important information regarding their sub-I, such as application material, duration of sub-I, and dates offered to interested applicants. Although VSAS serves as an invaluable online resource to applicants, many residency programs still provide the link to their website for more detailed and up-to-date information. Hence, residency program websites are still serving as an irreplaceable resource for residency applicants, further emphasizing the importance of having a comprehensive website for residency programs.

More importantly, with the recent proposed changes by National Board of Medical Examiners to the United States Medical Licensing Examination Step 1 from a numerical test score to pass/fail, residency programs will need to modify their resident selection process accordingly.[6] Without objective metrics such as test scores, it stands to reason that components of the application process such as sub-Is and interviews that provide an in-person interaction to applicants will play a more important role in resident selection. Consequently, it is likely that there will be an even increasing number of senior medical students who will apply to complete sub-Is at residency programs across the country. Residency programs should focus their efforts on improving the quality of online information available to applicants to ensure attracting the most qualified applicants.

This study is not without its limitations. Residency program websites are dynamic, and their contents are subject to frequent changes. In addition, the quality and availability of online information are somewhat dependent on the preferences of residency program leadership and technical skills of residency program coordinators to post relevant information online. The scoring system used in our study has not been validated and was used in only one study previously.[3] Moreover, additional information from the perspective of applicants is required to understand what information they consider essential on plastic surgery websites with regard to sub-Is to be included in this scoring system. As mentioned previously, residency program websites are not the only source of information available to applicants. Although we consider PSRW to be a central resource for medical students, other resources such as mentors, career advisors, online student forums, and VSAS allow senior medical students to make informed decisions about choosing programs to complete sub-Is.

CONCLUSIONS

PSRWs are sparse with information related to sub-Is. More than half of residency programs did not include any information related to sub-Is. In both univariate and multivariate analysis, program type was not found to be an important factor in including sub-I–related information on residency program websites. Larger residency programs in terms of number of residents and number of full-time faculty were associated with having more online information related to sub-I. Given the critical role of sub-Is in resident selection process for residency programs and in creating a competitive application for residency applicants, this evident lack of information seems detrimental to all parties involved. Hence, future efforts of all plastic surgery residency programs should be focused on updating their websites to provide comprehensive information regarding different components of the application process.