Accessibility and content of individualized adult reconstructive hip and knee/musculoskeletal oncology fellowship web sites.

BACKGROUND: Accessible, adequate online information is important to fellowship applicants. Program web sites can affect which programs applicants apply to, subsequently altering interview costs incurred by both parties and ultimately impacting rank lists. Web site analyses have been performed for all orthopaedic subspecialties other than those involved in the combined adult reconstruction and musculoskeletal (MSK) oncology fellowship match.
METHODS: A complete list of active programs was obtained from the official adult reconstruction and MSK oncology society web sites. Web site accessibility was assessed using a structured Google search. Accessible web sites were evaluated based on 21 previously reported content criteria.
RESULTS: Seventy-four adult reconstruction programs and 11 MSK oncology programs were listed on the official society web sites. Web sites were identified and accessible for 58 (78%) adult reconstruction and 9 (82%) MSK oncology fellowship programs. No web site contained all content criteria and more than half of both adult reconstruction and MSK oncology web sites failed to include 12 of the 21 criteria.
CONCLUSIONS: Several programs participating in the combined Adult Reconstructive Hip and Knee/Musculoskeletal Oncology Fellowship Match did not have accessible web sites. Of the web sites that were accessible, none contained comprehensive information and the majority lacked information that has been previously identified as being important to perspective applicants.

Introduction

Prior to the formalized fellowship match, interviews and position offers were uncoordinated and unregulated. This decentralized process hindered both parties from adequately vetting their alternatives [1]. To formalize these offers and create an equitable process, most specialties adopted a formal match, similar to what exists for the residency match. The American Association of Hip and Knee Surgeons (AAHKS), The Hip Society, The Knee Society, and the Musculoskeletal Tumor Society (MSTS) chose to establish a combined fellowship match for orthopaedic residents interested in subspecializing in adult reconstruction or musculoskeletal (MSK) oncology [2].

Without the added pressure of securing a position early in the application process, applicants and programs now go through a more structured process. This process has significant costs associated with it, as more interviews are conducted in search of an ideal match. It has been suggested that accessible, adequate online information could prevent unnecessary costs by assisting applicants compare programs before applying or interviewing 3, 4, 5. Evidence supporting how web-based information can impact residency and fellowship matches has been well summarized by past analytical reports of various subspecialties' web sites 6, 7, 8, 9 (B. L. Young et al, Unpublished results, 2017). In short, “literature shows that a program's web site can attract or deter applications, as well as impact applicants' rank lists” 10, 11, 12, 13, 14. Specifically, some orthopaedic “fellowship applicants valued a program's web site more than their peers' opinions of the program” [3].

Analyses have been performed to assess the accessibility and content of individualized program web sites of several orthopaedic subspecialty fellowships 6, 7, 8, 9, 15, 16 (B. L. Young et al, Unpublished results, 2017). To our knowledge, no such analysis has been performed for the web sites of programs participating in the Adult Reconstructive Hip and Knee/Musculoskeletal Oncology Fellowship Match. The purpose of this investigation is to perform this analysis.

Material and methods

This study was exempt from institutional review board approval. Data collection and accessibility analysis was performed on January 27, 2017. As this was in the middle of the interview season, it was felt that any annual updates to a web site would have been completed by this date. Adult reconstruction and MSK oncology fellowships were analyzed together because it is the only combined match in orthopaedics. Complete, separate lists of adult reconstruction fellowships and MSK oncology fellowships were found on the AAHKS and MSTS web sites, respectively 17, 18. Web site accessibility was based on its searchability using Google. Search phrases included all combinations of “program name” from the lists AND “adult reconstruction” OR “musculoskeletal oncology” AND “orthopaedic” AND “fellowship.” Only the first page of search results was viewed, similar to the search result sample size used in similar past studies 7, 8, 9 (B. L. Young et al, Unpublished results, 2017).

Web site content was assessed based on the criteria used in homologous analyses concerning other orthopaedic subspecialties 6, 7, 8, 9 (B. L. Young et al, Unpublished results, 2017). The fellowship web sites were analyzed for the inclusion of any information related to research opportunities and requirements, as well as current or past research performed by fellows. Logistical information such as a list of fellowship faculty, rotation schedules, on-call expectations, and case descriptions were also collected. Academic information analyzed included any mention of intra-institutional meetings (ie, grand rounds), journal clubs, conferences or meetings sponsored by program (ie, national and societal conferences), and teaching responsibilities of residents and medical students. Other pertinent information assessed was a list of current fellows, a list of previous fellows, previous education of current fellows (ie, medical school and residency), alumni career choices, description of the application process, program director and coordinator's contact information, fellow's salary, and a program description. Two authors performed independent web site reviews and reached a collective consensus when discrepancies arose in the collection of data.

Results

According to the program lists provided by the AAHKS and MSTS, there were 74 individual adult reconstruction fellowship and 11 individual MSK oncology fellowship programs 17, 18. One adult reconstruction program was listed twice and was subsequently counted as one program. A Google search for each program's fellowship web site found that 78.38% (58 of 74) of adult reconstruction fellowships and 81.82% (9 of 11) MSK oncology fellowships had accessible web sites.

The content of the 58 accessible adult reconstruction and 9 accessible MSK oncology fellowship web sites varied considerably. No web site contained all content criteria and more than half of both joint reconstruction and MSK oncology web sites failed to include 12 of the 21 criteria. Regarding adult reconstruction web sites, the 3 most available content items were program description (98.28%), case descriptions (96.55%), and research opportunities (89.66%). Regarding the MSK oncology fellowship web sites, the 3 most available content items were a description of the application process, program description, and research opportunities (all 100%). Accessible content for adult reconstruction and MSK oncology web sites is summarized in Tables 1 and 2.

Table 1

Depiction of proportion of 58 individual adult reconstructive hip and knee fellowship web sites that contained information pertaining to 21 content criteria.

Number of individual web sites	% (n = 58)
Program description	98.28 (57)
Case descriptions	96.55 (56)
Research opportunities	89.66 (52)
Description of application process	82.76 (48)
Research requirements	68.97 (40)
Institutional meetings	67.24 (39)
Attending faculty	65.52 (38)
Coordinator contact info	56.90 (33)
Teaching responsibilities	50.00 (29)
Fellow salary	34.48 (20)
National meetings sponsored	34.48 (20)
Rotation schedules	31.03 (18)
Journal clubs	31.03 (18)
Out-patient clinic expectations	25.86 (15)
Current fellows	25.86 (15)
Medical school and residency of current fellows	20.69 (12)
Director contact info	17.24 (10)
On-call expectations	13.79 (8)
Current and previous research	13.79 (8)
Previous fellows	13.79 (8)
Job choice of previous fellows	8.62 (5)

Table 2

Depiction of proportion of 9 individual adult reconstructive musculoskeletal oncology fellowship web sites that contained information pertaining to 21 content criteria.

Number of individual web sites	% (n = 9)
Description of application process	100.00 (9)
Program description	100.00 (9)
Research opportunities	100.00 (9)
Coordinator contact info	77.78 (7)
Research requirements	77.78 (7)
Fellow salary	66.67 (6)
Case descriptions	66.67 (6)
Institutional meetings	66.67 (6)
Attending faculty	66.67 (6)
National meetings sponsored	44.44 (4)
Teaching responsibilities	44.44 (4)
Current fellows	33.33 (3)
Rotation schedules	33.33 (3)
On-call expectations	33.33 (3)
Journal clubs	33.33 (3)
Director contact info	33.33 (3)
Current and previous research	11.11 (1)
Out-patient clinic expectations	11.11 (1)
Previous fellows	11.11 (1)
Medical school and residency of current fellows	11.11 (1)
Job choice of previous fellows	11.11 (1)

Discussion

The formalized adult reconstructive hip and knee and MSK oncology fellowship match is unique in that it is the only combined match, catering to 2 separate orthopaedic subspecialties. Past analyses of fellowship specific web sites for spine, hand, sports medicine, pediatric orthopaedics, shoulder and elbow, orthopaedic trauma, and foot and ankle have demonstrated that inadequacies exist in their online accessibility and content 6, 7, 8, 9, 15, 16 (B. L. Young et al, Unpublished results, 2017). The joint reconstruction and MSK oncology web site data herein completes the analytical series for the field of orthopaedics and identifies notable web site limitations.

In the American Academy of Orthopaedic Surgeons (AAOS) publication, “Considerations in Choosing a Fellowship,” applicants are directed to consider if the fellowship is involved with an associated residency program, conferring an environment of formal teaching and conferences [19]. However, the analysis herein found that many adult reconstruction and MSK oncology fellowship web sites failed to provide relevant information to this direction such as fellow teaching responsibilities, journal clubs, meetings sponsored by the fellowship program, or institutional meetings. Also, the AAOS urges applicants to consider the fellowship's research activity and requirements, as well as the balance of resident and fellow responsibilities [19]. Our investigation found that many web sites lacked information about their program's research requirements, and very few divulged the research of current and previous fellows. Although applicants are urged to consider the balance of resident and fellow responsibilities, few adult reconstructive and MSK oncology fellowship web sites included information about on-call expectations and out-patient clinic expectations. These findings are consistent with content reviews of other orthopaedic fellowship web sites and suggest the need for leadership to consider standardization of web site information.

In Bernstein's “Factors to Consider when Considering a Fellowship,” he surmises that the most important aspect of a training program is that it has successfully prepared a fellow for a career similar to that desired by the prospective applicant [20]. However, our analysis found that the answer to this fundamental aspect is not readily accessible on many programs' web sites. For instance, less than 14% of adult reconstruction program web sites provided a list of their previous fellows or information about the job choice of their previous fellows. This same shortcoming was seen among MSK oncology fellowship web sites. Although program-specific information can be gathered during interviews, several recent reports have highlighted the significant interview costs incurred by applicants to both residency and fellowship programs. A 2015 survey of orthopaedic fellowship applicants found that applicants spent an average of $5874 and 49% of applicants had to borrow money to cover interview expenses [5]. Moreover, the authors found that 86% of applicants missed 8 days of residency training, leading 62% of residency program directors to label the fellowship interview process as “extremely disruptive” to their program. Survey respondents also requested that fellowship programs provide updated web sites with standardized information, which the authors discuss could help limit the need for an interview to gather important information. A separate study evaluating the interview costs incurred by the fellowship programs found that programs incurred 65 hours of opportunity cost (eg, the cost of faculty participating in the interview process rather than seeing patients in clinic or operating) and upwards of $4500 in spending during interviews [3]. Extra attention should be drawn to the expenditures of applicants participating in the Adult Reconstructive Hip and Knee/Musculoskeletal Oncology Fellowship Match as a growing number of orthopaedic trainees are completing multiple fellowships, especially a combination of adult reconstruction and MSK oncology [21]. There is no question that the financial and time burden placed on programs and applicants by the current system is not optimal. Simple efforts, such as providing an easily accessible and comprehensive online information source, could potentially increase application selectivity and decrease expenditures and opportunity costs by all parties involved.

Online databases could serve as an acceptable and more manageable alternative to individualize web sites, by providing detailed program characteristics to prospective applicants who wish to browse and compare all their options. The AAOS offers such a database, the Postgraduate Orthopaedic Fellowships online directory. However, it should be mentioned that we performed a quick overview of the directory and found several discrepancies between it and the official program rosters found on AAKHS and MSTS societal web sites. For example, the number of programs differed for both subspecialties depending on which resource was queried, and several programs were listed under different official names, making it difficult to aggregate information between sources. Another shortcoming of the AAOS directory is that it is available only to AAOS members or non-members for purchase, inherently limiting the directory as a resource for applicants without access [19]. This current lack of a free, comprehensive database increases the importance and utility of individual adult reconstructive hip and knee and MSK oncology fellowship web sites in order to recruit applicants and provide important information. On the other hand, the development of a comprehensive and accessible database may be a solution for the inadequate web sites discussed in this study, helping programs with web sites of inferior content and accessibility avoid failure to recruit applicants who rely on individual program web sites for information.

Past literature gives insights into the quality of individualized program web sites of other orthopaedic subspecialties. The data points assessed in this study were similar to other prior studies reflecting consistent information desired by fellowship applicants 6, 7, 8, 9, 15, 16, 22 (B. L. Young et al, Unpublished results, 2017). In the sports medicine literature, there have been 2 studies that evaluated the accessibility and content of accredited orthopaedic sports medicine fellowship web sites, Mulcahey et al in 2013 with an update by Yayac et al in 2017 7, 22. The follow-up study failed to demonstrate improved web site accessibility and found continued deficits in web site content, showing that many sports medicine fellowships continue to underutilize the Internet as a tool that potentially could improve the match process for all parties [22]. Adult reconstruction and MSK oncology fellowships should be aware of this failure to improve web site accessibility and content to avoid a similar pit fall. It is our perspective that the 21 content criteria domains used in this assessment (Tables 1 and 2) can serve as a defined yet flexible outline for guiding content of web sites for orthopaedic fellowships.

This study contains limitations inherent to web site analyses. First, the analysis is a snap shot in time of web site content and accessibility. Subsequent updates to the web site content and accessibility may occur. When assessing accessibility, only the first page of search results was viewed. Although the web sites presence on the first page of search results is an objective measure that is similar to the protocols of past studies 7, 8, 9 (B. L. Young et al, Unpublished results, 2017), there is an unlikely chance that a web site was present on subsequent pages of search results. Web site content was judged as present or absent only. Therefore, some content may have been recorded as present when it lacks the quality to sufficiently answer applicant queries. However, any presence of the second limitations would only increase the evidence supporting our conclusion that the web sites lacked comprehensive, adequate information.

Conclusions

In conclusion, 22% of adult reconstruction and 18% MSK oncology fellowship programs did not even have a functioning web site to provide basic program information. Of those with web sites, none included all informative content deemed important to applicants. These inadequacies may hinder applicant recruitment, prevent programs and applicants from avoiding unnecessary interview expenditures, and lead to sub-optimal matches. Although programs with the strongest reputations may not be directly impacted by a weak web presence, the majority of programs, and specifically newer or more remotely located programs, will likely be most negatively impacted by a limited web site.