US Residents' Perspectives on the Introduction, Conduct, and Value of American Board of Anesthesiology's Objective Structured Clinical Examination-Results of the 1st Nationwide Questionnaire Survey.

INTRODUCTION: Passing the Objective Structured Clinical Examination (OSCE) is currently a requirement for the vast majority (not all) of candidates to gain American Board of Anesthesiology (ABA) initial certification. Many publications from the ABA have attempted to justify its introduction, conduct and value. However, the ABA has never attempted to understand the views of the residents.
METHODS: A total of 4237 residents at various training levels from 132 programs were surveyed by asking to fill a Google questionnaire prospectively between March 8th, 2021 and April 10th, 2021. Every potential participant was sent an original email followed by 2 reminders.
RESULTS: The overall response rate was 17.26% (710 responses to 4112 invitations). On a 5-point Likert scale with 1 as "very inaccurate" and 5 as "very accurate," the mean accuracy of objective structured clinical examination (OSCE) in assessing communication skills and professionalism was 2.3 and 2.1 respectively. In terms of the usefulness of OSCE training for improving physicians' clinical practice, avoiding lawsuits, teaching effective communication with patients and teaching effective communication with other providers, the means on a 5-point Likert scale with 1 as "Not at all useful" and 5 as "Very useful" were 1.86, 1.69, 1.79, and 1.82 respectively. Residents unanimously thought that factors such as culture, race/ethnicity, religion and language adversely influence the assessment of communication skills. On a 5-point Likert scale with 1 as "not at all affected" and 5 as "very affected," the corresponding scores were 3.45, 3.19, 3.89, and 3.18 respectively. Interestingly, nationality and political affiliation were also thought to influence this assessment, however, to a lesser extent. In addition, residents believed it is inappropriate to test non-cardiac anesthesiologists for TEE skills (2.39), but felt it was appropriate to test non-regional anesthesiologists in Ultrasound skills (3.29). Lastly, nearly 80% of the residents think that money was the primary motivating factor behind ABA's introduction of the OSCE. Over 96% residents think that OSCE should be stalled, either permanently scrapped (60.8%) or paused (35.8%).
CONCLUSIONS: Anesthesiology residents in the United States overwhelmingly indicated that the OSCE does not serve any useful purpose and should be immediately halted. Copyright:

INTRODUCTION

As the certifying body for anesthesiologists since 1938, the American Board of Anesthesiology (ABA), through its team of dedicated anesthesiologist volunteers and staff, administers, supports initial and subspecialty certification assessments as well as continuing certification programs. The ABA claims to “promote lifelong learning with a commitment to quality clinical outcomes and patient safety” although there is no evidence to support such an assertion.[1]

In 2018, the ABA became the first and only one of the 24 member boards of the American Board of Medical Specialties (ABMS; Chicago, IL, USA) to introduce the objective structured clinical examination (OSCE) as part of its examination for initial certification. Professionalism and interpersonal and communication skills are two elements of the six competency-based medical education attributes advocated by the Accreditation Council for Graduate Medical Education (ACGME) and tested in the OSCE. The other four tested attributes are patient care, medical knowledge, systems-based practice, and practice-based learning.[2] In part, the OSCE was introduced to satisfy the 2015 ACGME safety and quality improvement requirements for all residency programs, although ACGME itself did not specify that the OSCE should be used to test these requirements. In fact, the ACGME did not mandate any testing at the certification point. However, this does not mean that the ABA should not implement meaningful changes in an effort to serve the public interest by advancing the standards of anesthesiology practice through certification. Although nonprofit, the ABA is a private organization, and the certification is voluntary. Nonetheless, these changes should be evidence based, fair, and practical.

The OSCE implementation may have been a step in the right direction. In a recent publication, Warner et al. observed that the OSCE tested certain elements of the ACGME's core competencies that were not tested by the remaining components of the ABA examination.[3] However, an accompanying editorial and a letter to the editor that followed the publication were not as enthusiastic.[45]

Our study aims to understand the perspectives of residents on the introduction, conduct, and value of the OSCE as part of the applied examination of the ABA. It is hoped that the ABA will take these residents' views into consideration in their future plans regarding the continuation or modification of this component of their certification examination. In addition, other member boards of the ABMS might use the results of this survey in their own planning. In this regard, it should be noted that the Federation of State Medical Boards and National Board of Medical Examinations, the co-sponsors of the United States Medical Licensing Examination®, announced the discontinuation of work to relaunch a modified Step 2 clinical skills assessment examination (an OSCE equivalent).[6]

METHODS

The institutional review board of the University of Pennsylvania approved this study on March 8, 2021.

The potential participants were the residents and fellows currently enrolled in various anesthesiology programs including combined programs (such as pediatrics–anesthesiology). Considering that the entire study was performed between March 8, 2021, and April 10, 2021, the participants were residents of the years 2021, 2022, 2023, and 2024 and fellows. 2021 refers to the academic year 2020–2021. The graduating class of 2021 would have started their residency in 2017 (academic year 2017–2018).

The ABA has performed various questionnaire surveys involving anesthesiology residents comprising several aspects of their training. The aim of this study was to encompass aspects that directly influence the residents that were not explored by the ABA. Although the OSCE was introduced in 2018 and had played a significant role in the lives of residents, the ABA had not performed any surveys to gather their opinion in this regard. The areas that were studied are assessment in the OSCE, format of the OSCE, value of the OSCE, factors affecting the communication section of the OSCE, possible motive for the OSCE introduction, and the future of the OSCE. The questions asked under various headings are summarized in Appendix 1.

E-mails with a link to a Google Form containing the questions listed in Appendix 1 and a selection of responses were sent to 4237 anesthesiology residents enrolled in 132 programs across the USA. The calculated sample size for a population size of 4237, to keep a margin of error of 4% and a confidence level of 95%, was 526. The response rate in online E-mail surveys is generally 5%–10%. In addition, we could not be certain of the E-mails likely to be blocked, nondelivered, etc., As a result, we decided to invite all the residents whose E-mails could be obtained.

The name and training year of the resident were obtained from the institutional program website. Majority of the E-mails were obtained from the program websites and the remaining from the member's section of the American Society of Anesthesiology website. Not all the E-mails were listed; as a result, some residents were left out from the survey. The first batch of E-mails was dispatched between March 8, 2021, and March 17, 2021. A second request to all the residents in the mailing list was sent between March 17, 2021, and March 27, 2021. Two residents requested their name to be removed from the list, and we obliged. A third and final reminder was sent between between March 28, 2021, and April 4, 2021. The form stopped receiving responses on April 10, 2021, at 10:00 p.m. The last E-mail mentioned the current response rate and total responses. Research has shown that providing this information increases the response rates.[7] The respondents' E-mail address was not recorded. The questions were formulated by the corresponding author, vetted by two other faculty members of the Perlman School of Medicine, and approved by the vice chair for research. Different answer options were used for different questions/group of questions. However, most responses were on a Likert scale (which assumes the strength of association as linear). Participation was entirely voluntary. By participating in the questionnaire survey, the residents gave consent to participate in the study. All questions were compulsory. The respondents could not proceed to the next page without answering all the questions in each page and could not submit the form without answering every question. The Google Form was set to prevent multiple responses from a single E-mail address after the 1st batch of E-mails was dispatched. E-mails were sent individually to each resident (instead of residency program directors or program coordinators) to increase the response rate. Residents at Penn Medicine (the institution of the corresponding author) were excluded to eliminate possible bias.

The responses were analyzed on April 11, 2021, with mean and standard deviation along with median values being reported as calculated from the 5-point Likert scale equivalents.

Appendix 2 provides the list of residency programs in the United States to which the questionnaire was E-mailed.

RESULTS

A total of 4237 E-mails were dispatched to various anesthesiology residents across the country. Of these, 4112 were delivered, whereas 125 bounced back for a variety of reasons (Iron Port E-mail Security Appliance, remote server returns, and undeliverable E-mails). In addition, a total of three residents requested their names to be removed from the mailing list. Three attempts were made to obtain responses, with each E-mail reminder being sent for the following time periods: March 8, 2021–March 17, 2021 (1st attempt), March 17, 2021– March 27, 2021 (2nd attempt), and March 28, 2021–April 4, 2021 (3rd and final attempts). The form stopped receiving responses on April 10, 2021, at 10:00 p.m., with a grand total of 710 responses being recorded (17.26%). The total included 284, 167, and 259 responses during each of the respective time periods. Table 1 summarizes the responses by the timing of E-mails sent along with the average response percentage for each.

Table 1

Responses by the attempts

Dates of E-mail dispatch	Number of residents to whom E-mails were dispatched	Number of E-mails bounced/not delivered	Number of responses received	Overall percentage response
March 08, 2021-March 17, 2021 (1st attempt)	4237	125	284	6.70
March 17, 2021-March 26, 2021 (2nd attempt)	4236	124	167	3.94
March 27, 2021-April 04, 2021 (3rd and final reminders)	4235	123	259	6.11

Of the 710 anesthesiology residents who took part in the study, the majority who answered were from CA (clinical anesthesia training) 2 and CA3. The average response percentage was highest among fellows (36.51%), however only 63 E-mails were sent to this group. Respectively, 583, 1153, 1234, 1204, and 63 E-mails were sent to trainees in postgraduate year 1, CA1, CA2, CA3, and fellows. The distribution of total responses was 47 (8.06%), 146 (12.66%), 212 (17.18%), 282 (23.42%), and 23 (36.51%) among these cohorts, respectively [Table 2].

Table 2

Percentage response for each training year

Training category	Number of E-mails presumably delivered, opened, and read	Complete responses received	Percentage response
Postgraduate year 1	583	47	8.06
CA training 1	1153	146	12.66
CA training 2	1234	212	17.18
CA training 3	1204	282	23.42
CA training 4	63	23	36.51
Total	4237	710	16.75

CA=Clinical anesthesia

In addition, the responses varied greatly by the state of program location. In terms of response distribution by state, Minnesota, Puerto Rico, and Connecticut had the best response rates of 62.5% (15/24), 54.55% (6/11), and 50.91% (28/55), respectively. The lowest three states were New Jersey, Iowa, and Oregon at 0% (0/28), 5.35% (3/56), and 5.71% (2/35), respectively. Table 3 summarizes the response percentages for each state/territory to which E-mails were sent.

Table 3

Responses by state - percentage and total

State	Total E-mails presumed to be delivered, opened, and read	Total responses received	Response in percentage
Alabama	60	12	20.00
Arizona	44	15	34.09
Arkansas	33	11	33.33
California	360	70	19.44
Colorado	50	10	20.00
Connecticut	55	28	50.91
Florida	282	48	17.02
Georgia	81	17	20.99
Illinois	198	25	12.63
Indiana	74	10	13.51
Iowa	56	3	5.36
Kansas	52	4	7.69
Kentucky	21	3	14.29
Louisiana	103	8	7.77
Maine	18	4	22.22
Maryland	54	7	12.96
Massachusetts	297	69	23.23
Michigan	70	13	18.57
Minnesota	24	15	62.50
Mississippi	34	3	8.82
Missouri	120	12	10.00
Nebraska	51	17	33.33
New Hampshire	20	3	15.00
New Jersey	28	0	0.00
New Mexico	25	3	12.00
New York	572	63	11.01
North Carolina	126	25	19.84
Ohio	188	25	13.30
Oklahoma	44	4	9.09
Oregon	35	2	5.71
Pennsylvania	196	33	16.84
Puerto Rico	11	6	54.55
Rhode Island	31	2	6.45
South Carolina	62	12	19.35
Tennessee	97	20	20.62
Texas	200	34	17.00
Utah	26	3	11.54
Vermont	16	2	12.50
Virginia	96	11	11.46
Washington State	128	15	11.72
Washington D.C	47	9	19.15
West Virginia	28	5	17.86
Wisconsin	124	29	23.39
Total	4237	710	16.75

Table 4 summarizes the questions asked from the survey as well as the response percentage for each category of the 5-point Likert scale (1 as “very inaccurate” and 5 as “very accurate”). Pie charts of these responses may be found at http://dx.doi.org/10.13140/RG.2.2.25924.55680. Mean and standard deviation along with median values for each question were also reported. Responses regarding the accuracy of the OSCE examination in assessing communication skills were reported as largely inaccurate, as 24.5% and 30.8% responded with “very inaccurate” and “inaccurate,” respectively. The mean and median values (2.33 and 2, respectively) also support this finding. In addition, the residents were asked whether they felt the OSCE assessed professionalism accurately. Responses for this aspect of the examination were received even more poorly, as the mean value was reported at 2.14 (1.00).

Table 4

Questions asked in the survey, and the response percentage for each category of the 5-point Likert scale with their mean and median

Component and questions	Scoring scales and responses in percentage of the total respondents					Mean (SD)	Median
Assessment in the OSCE	1	2	3	4	5
How accurate do you believe the OSCE is in assessing communication skills? (very inaccurate, inaccurate, neutral, accurate, very accurate)	24.5	30.8	32.3	11.4	1.0	2.33 (1.00)	2
How accurate do you believe the OSCE is in assessing professionalism? (very inaccurate, inaccurate, neutral, accurate, very accurate)	32.3	32.8	24.5	9.6	0.8	2.14 (1.00)	2
Format of the OSCE	1	2	3	4	5
How appropriate do you believe it is for the OSCE to assess noncardiac anesthesiologists in TEE? (very inappropriate, inappropriate, neutral, appropriate, very appropriate)	24.1	33.5	24.8	14.2	3.4	2.39 (1.10)	2
How appropriate do you believe it is for the OSCE to assess nonregional anesthesiologists in US? (very inappropriate, inappropriate, neutral, appropriate, very appropriate)	10.8	14.5	24.5	34.9	15.2	3.29 (1.21)	2
Value of the OSCE	A	B	C	D	E
How useful do you believe OSCE training is for improving physicians’ clinical practice?	47.5	30.7	12.0	7.7	2.1	1.86 (1.04)	2
How useful do you believe OSCE training is for avoiding lawsuits?	53.2	29.9	12.4	3.9	0.6	1.69 (0.88)	1
How useful do you believe the OSCE training is for teaching physicians how to effectively communicate with patients?	52.4	28.0	9.61	8.6	1.4	1.79 (1.02)	1
How useful do you believe OSCE training is for teaching physicians how to effectively communicate with other providers?	52.7	25.2	11.3	9.0	1.8	1.82 (1.07)	1
Factors affecting the communication section of the OSCE	A	B	C	D	E
To what extent do you think that the OSCE’s assessment of communication skills is affected by the following characteristics of the test taker?
Culture	8.7	11.3	28.9	28.9	22.3	3.45 (1.20)	4
Race/ethnicity	12.1	18.5	24.8	27.5	17.2	3.19 (1.26)	3
Religion	28.9	34.4	15.9	9.7	11.1	3.89 (1.30)	4
Language	4.9	6.2	20.0	32.4	36.5	3.18 (1.12)	3
Presumed nationality	11.8	18.7	27.2	23.9	18.3	2.40 (1.26)	2
Presumed political affiliation	32.8	34.9	15.8	8.2	9.7	2.28 (1.26)	2
Motive of the ABA with regard to the OSCE introduction	I	II	III	IV
What do you think is the main motive of the ABA to start OSCE?	78.9	14.1	0.7	6.3
Future of the OSCE	IA	IB	IC	ID
What do you think should be the future of the OSCE?	60.8	32.7	3.1	3.4

1=Very inaccurate, 2=Inaccurate, 3=Neutral, 4=Accurate, 5=Very accurate, A=Not at all useful, B=Of very little use, C=Neutral, D=Somewhat useful, E=Very useful, a=Not at all affected, b=Minimally affected, c=Somewhat affected, d=Affected, e=Very affected, I=Financial gain to the ABA, II=Improve patient care, III=Increase patient satisfaction, IV=Decrease disciplinary proceeding against anesthesiologists by the licensing bodies, IA=Permanently scrapped, IB=Paused pending demonstration that it improves patient outcomes, IC=Paused pending demonstration that it decreases disciplinary proceeding against anesthesiologists by the licensing bodies, ID=Should continue as is. OSCE=Objective structured clinical examination, ABA=American Board of Anesthesiology, TEE=Transesophageal echocardiogram, US=Ultrasound

The next portion of the survey asked anesthesiology residents to rank the appropriateness of different portions of the OSCE examination. The first of which asked whether residents believed it is appropriate for the OSCE to assess noncardiac anesthesiologists in transesophageal echocardiogram (TEE). Many of the respondents indicated that such an examination topic was inappropriate, as 24.1% and 33.5% responded “very inappropriate” and “inappropriate,” respectively. The calculated mean (2.39) and median (2.0) values also support the finding that this portion was believed to be inappropriate. Interestingly, the resident anesthesiologists believed that the OSCE's portion assessing the nonregional anesthesiologists in ultrasound (US) was somewhat appropriate, as the mean value was relatively better at 3.29 (1.21).

The residents were then asked a series of questions meant to determine the perceived value of OSCE training and assessment. A 5-point Likert scale was also used for analyses of this section, with responses ranging from “not at all useful”[1] to “very useful.”[5] As far as the OSCE's usefulness for improving physicians' clinical practice, 47.5% and 30.7% of the residents indicated that the OSCE was “not at all useful” and “of very little use,” respectively. The mean value for this question was also very low at 1.86 (1.04). The perceived usefulness of the OSCE in terms of avoiding lawsuits was reported even lower, with a mean value of 1.69 (0.88). In addition, the residents were asked how useful they believe OSCE training was for effectively communicating with patients as well as other providers. Both of these topics from the OSCE were regarded as of little use, with mean values being 1.79 (1.02) and 1.82 (1.07), respectively.

Communication skills are tested very widely in the OSCE examination, be with other physicians and providers or patients. The residents were asked to rank on a 5-point Likert scale (1 being “not at all affected” to 5 being “very affected) whether different categories such as culture, race/ethnicity, religion, language, presumed nationality, and presumed political affiliation affected their ability to perform in the communication section of the OSCE. Respectively, the mean values for each of the aforementioned categories were 3.45 (1.2), 3.19 (1.26), 3.89 (1.30), 3.18 (1.12), 2.40 (1.26), and 2.28 (1.26), indicating that the residents are in agreement that the OSCE's assessment of communication skills is affected by external factors. Presumed nationality and political affiliation were believed to be least affected, yet the means were very low (2.40 and 2.28, respectively).

In general, the purpose and conduct of the OSCE component of the ABA applied examination was viewed very poorly by the majority of respondents. A total of 560 of the 710 (78.9%) respondents agreed with the assumption that the main purpose of introducing the OSCE was to increase the ABA revenue. Table 4 provides full response in percentages to various statements posed to the respondents. In addition, the respondents were asked to choose one of the four possible options for what they think should be the future of the OSCE. Of the 710 respondents, 432 (60.8%) believed that the OSCE should be permanently scrapped. Furthermore, 32.7% and 3.1% believed that the OSCE examination should be paused pending demonstration that it improves patient outcomes and decreases disciplinary proceedings against anesthesiologists, respectively. Only 24 respondents (3.4%) expressed that the OSCE should continue as is.

Of the 4112 E-mails sent, a total of 710 residents responded after three attempts to obtain responses (17.26%). Figures 1-8 summarize the response trends for select questions from the survey questionnaire, demonstrating how the distribution of responses remained largely consistent throughout the entire duration of the study. Such a finding adds credibility to the extrapolation of our findings to the larger resident anesthesiology community. Figure 9 represents the residents' responses to the questions related to factors that can produce potential bias in the conduct of the OSCE in a bar graph.

Figure 1

Percentage response at the end of every 100th response to display the trend – answer to the question “How accurate do you believe the objective structured clinical examination is in assessing communication skills?”

Figure 8

Percentage response at the end of every 100th response to display the trend – answer to the question “What do you think should be the future of the objective structured clinical examination?”

Figure 9

Extent of the effects of culture, race/ethnicity, language, nationality, religion, and political affiliation on the objective structured clinical examination assessment of communication skills

Percentage response at the end of every 100th response to display the trend – answer to the question “How accurate do you believe the objective structured clinical examination is in assessing professionalism?”

Percentage response at the end of every 100th response to display the trend – answer to the question “How useful do you believe objective structured clinical examination training is for improving physicians' clinical practice?”

Percentage response at the end of every 100th response to display the trend – answer to the question “How useful do you believe objective structured clinical examination training is for avoiding lawsuits?”

Percentage response at the end of every 100th response to display the trend – answer to the question “How useful do you believe the objective structured clinical examination training is for teaching physicians how to effectively communicate with patients?”

Percentage response at the end of every 100th response to display the trend – answer to the question “How useful do you believe objective structured clinical examination training is for teaching physicians how to effectively communicate with other providers?”

Percentage response at the end of every 100th response to display the trend – answer to the question “What do you think is the main motive of the ABA to start objective structured clinical examination?”

Table 5 contains the list of abbreviations used in this paper.

Table 5

Abbreviations used in the text

OSCE - Objective structured clinical examination

ABA - American Board of Anesthesiology

ABMS - American Board of Medical Specialties

ACGME - Accreditation Council for Graduate Medical Education

CSA - Clinical skills assessment examination

USMLE - United States Medical Licensing Examination

NBA - National Board of Medical Examinations

FSMB - Federation of State Medical Boards

CA - Clinical anesthesia

TEE - Transesophageal echocardiogram

US - Ultrasound

SP - Standardized patients

SOE - Structured oral examination

MOCA - Maintenance of Certification in Anesthesiology

DISCUSSION

We made significant efforts to increase response rates. “respondent fatigue” is a major impediment to getting high response rates in online surveys.[8] We might have made the situation worse by compelling the respondents to answer all the questions. Even in the ABA conducted surveys (which could be considered as an internal survey), the overall response rate was 36%.[910] External surveys with no participation incentive are known to produce even lesser response rates, about 10%.[1112] Several meta-analyses have revealed that web surveys generally get a 6%–15% lower response rate compared to other survey modes, with many studies conducted among students reporting response rates below 20%.[7] As a result, our overall response rate of 17.26% may be considered good especially given the lack of incentive and a relatively longer questionnaire. In addition, our survey could only be taken by those who were exposed to mock OSCE sessions. In fact, the corresponding author received E-mails from individual residents to that effect. With the ongoing pandemic, residents have limited opportunity to attend courses and departments have similar constraints in organizing OSCE training sessions. This is clearly indicated in increasing response rates among higher training year residents who might have had some exposure to mock OSCE before the pandemic.

Our survey explores an aspect of OSCE which is not studied by the ABA. Our results might indicate that the ABA has failed to communicate and convince the residents that the skills tested in the OSCE make a positive impact on their practice. It is also possible that such evidence might not exist. It seems that the major impetus for the ABA to begin the OSCE was monetary gain as opined by over 80% of the respondents. We hesitate to state that a cursory look at the ABA's tax returns gives credence to that belief. Being a nonprofit-making organization, these tax returns are available to the public (https://projects.propublica.org/nonprofits/organizations/60646523, Figure 10). The ABA program services revenue (in millions of US dollars) increased from 3.3 in 2001 to 15.6 in 2018. We do not know the impact of the OSCE on the revenue yet as it was only introduced in 2018. More research is underway to understand the perceptions of board-certified anesthesiologists participating in the Maintenance of Certification in Anesthesiology minute. If the feeling is widespread, it might have implications on the nonprofit status of the ABA. Nonprofit organizations are allowed to make profit, however they cannot design and implement policies for such purpose.

Figure 10

The American Board of Anesthesiology's revenue and expenses from 2001 to 20018 _source-https://projects.propublica. org/nonprofits/organizations/60646523)

The communication section is a major component of the OSCE. A majority of the residents think that the OSCE's assessment of communication skills is affected by factors such as candidate's culture, race/ethnicity, language, and nationality. Among these factors, race/ethnicity, culture, religion, and language are thought to have the highest sway. Unlike TEE and US components, this section is bound to be subjective and inclined to be affected by the implicit biases of the examiners and the standardized patients (SPs). Racial profiling is well known among physicians and dentists. In an interesting study, Patel et al. found that dentists are much more likely to recommend tooth extraction to a Black patient versus root canal work for a White patient for similar conditions, based on the race alone.[13] Black patients are seen by doctors as opioid seeking.[14] South Asian doctors are often presumed to have been trained in a foreign medical school.[15] While writing in the journal “Advances in Health Sciences Education,” Sharma and Kuper talked about the racism in our classrooms, educational and research institutions, and communities.[16] Graf et al. compared the students' (self-perception) and SPs' (external perception) perception of communication skills using uniform questionnaires in the context of an OSCE examination. They found significant gender difference in favor of female students performing better in the dimensions of empathy, structure, verbal expression, and nonverbal expression. In fact, male students deteriorated across all dimensions in the external perception between 2011 and 2014.[17] Recent ABA publications that attempted to validate the introduction of the OSCE have ignored any discussion of bias.[1819] Our results indicate that the unconscious bias training (which is provided by the ABA) may not remove the enormous suspicion held by over 90% of the residents. The futility of unconscious bias training in eliminating bias, racism, and discrimination is discussed by multiple authors.[202122] Considering that five OSCE stations test the communication elements, this is a huge challenge which is difficult to overcome. Nonetheless, it has a major bearing on the results. It is heartening to note that the ABA is concerned with racism in public health and has recently issued a statement in this regard.[23]

A large number of residents think that it is inappropriate to assess noncardiac anesthesiologists in TEE skills. This is entirely understandable. These skills typically help intensivists and cardiac anesthesiologists in patient management. Like the common adage “you use it or lose it,” a practitioner is bound to lose the skills without regular use. Memorizing the videos and spending few days to learn these skills just to pass the examination is probably not the best way the residents can use their time and resources. Residents were more divided regarding testing for US skills, with a small majority agreeing that it is appropriate to assess nonregional anesthesiologists in US.

More concerning is the strong perception among the residents that OSCE training does not teach them to effectively communicate with either patients or other health-care providers. It is common knowledge that we all individually tailor our approach to patients and modify our body language, demeanor, tone and degree, and depth of explanation based on their anxiety levels, knowledge, and understanding. It takes many years, perhaps even decades for us to perfect this art, and not all of us can achieve the same level of competency. In fact, it is counterproductive to patient care if we approach all patients in a standardized fashion.

The majority of the residents strongly disagree that OSCE training improves their clinical practice or helps in avoiding lawsuits. There is no concrete evidence to support the idea that board certification itself will either increase patient safety or decrease disciplinary actions from the licensing bodies.[2425] As an indicator of its certification cogency, the ABA used the license disciplinary actions as an indicator of written and oral specialty certification examination effectiveness.[24] Although negligence and incompetence are the most common causes, factors such as alcohol and substance abuse, inappropriate prescribing practices, inappropriate contact with patients, and fraud are responsible for the majority of such actions, and these cannot be foreseen in the OSCE or structured oral examination.[25] In a study that looked at the effectiveness of board certification as a surrogate indicator of provider competence, Silber et al. examined Medicare claims records of 144,883 patients in Pennsylvania who underwent general surgical or orthopedic procedures between 1991 and 1994.[26] The authors only looked at midcareer anesthesiologists and found a slightly higher mortality rate within 30 days of admission and failure-to-rescue rate among nonboard-certified anesthesiologists. However, they very importantly noted that nonboard-certified anesthesiologists tended to work in less equipped hospitals. The mortality numbers were very low, and death can be ascribed to too many factors, and, as a result, it is impossible to provide a a verdict one way or another. However, what is true is that an inability to obtain certification has enormous psychological, social, and financial consequences.[27] Failing initial board certification examinations has been reported as contributing to physician suicide.[28] Considering that the ABA found a high prevalence of burnout, distress, and depression among residents, it is important to take measures to reduce them.[10] In a system where institutional support, work–life balance, strength of social support, workload, and student debt are impacting physician well-being, it is vital that the ABA removes untested layers of the multilayer anesthesiology initial board examinations, instead of increasing them. The ABA certification is not demonstrated to save any lives; however, failure to get certification can cost the lives of anesthesiologists.

Our research is likely to provide much-needed feedback to the ABA about the residents' perception of the OSCE. Based on these results, the ABA might consider performing a larger study to get the views of the majority of residents; although, it is unlikely to get a different result. We followed the trends in the answers throughout the study. We did not notice any difference as the study progressed. It is essential for the ABA to hire an independent agency to administer such questionnaire surveys. Administration of biased examination (going by the majority of resident's opinion) to grant certification might compromise public health (instead of improving) by eliminating good clinicians. The necessary hurdles for certification should be evidence based and must serve the public interest.

The residents are most unified with regard to the future of the OSCE. Nearly 97% of the residents would like to see OSCE stalled; either permanently scrapped (61.3%) or paused pending demonstration that it improves patient outcomes (32.4%) or paused pending demonstration that it decreases disciplinary proceedings against anesthesiologists by licensing bodies (3.2%). Clearly, the ABA wanted to make a positive influence both in training and practice by introducing OSCE. Nevertheless, it seems to have failed.

Limitations of the study

Web-based surveys suffer from poor response rates, and our survey is no exception. Yet, a low margin of error for a 95% confidence augurs well. Nonopened E-mails, reluctance on the part of the E-mail recipient, forgetfulness, and indefinite postponement are some of the potential contributing factors. The corresponding author (who personally dispatched all the E-mails after work and during weekends) noticed that the responses typically came while the E-mails were being sent. However, as pointed out by Cook et al., “response representativeness is more important than response rate in survey research.”[29] Our survey represents 42 states, 80 programs, and residents of all training years. They are also overwhelmingly swung in one direction, which increases the strength of the results. As a result, we believe that the results are true representation of the mood and opinion of the residents across the nation.

CONCLUSIONS

Residents unanimously view negatively the introduction, conduct, and value of the OSCE component of the ABA certifying examination. They agree that ABA should abolish the OSCE. We are certain that as an organization that values resident's opinion very highly, the ABA will permanently scrap the OSCE with immediate effect.

We understand the sensitive nature and possible implications of the findings of our study. To maintain transparency, we will let anyone including the ABA to scrutinize our data after giving us adequate notice.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Questions and response options posed to the residents and fellows

Component and questions
Assessment in the OSCE (very inaccurate, inaccurate, neutral, accurate, very accurate)
How accurate do you believe the OSCE is in assessing communication skills?
How accurate do you believe the OSCE is in assessing professionalism? (very inaccurate, inaccurate, neutral, accurate, very accurate)
To what extent do you think that the OSCE’s assessment of communication skills is affected by the following characteristics of the test taker? (not at all affected, minimally affected, somewhat affected, affected, very affected)
Culture
Race/ethnicity
Language
Nationality
Religion
Political affiliation
Format of the OSCE? (very inappropriate, inappropriate, neutral, appropriate, very appropriate)
How appropriate do you believe it is for the OSCE to assess noncardiac anesthesiologists in TEE?
How appropriate do you believe it is for the OSCE to assess nonregional anesthesiologists in US?
Value of the OSCE (very useful, somewhat useful, neutral, of very little use, not at all useful)
How useful do you believe OSCE training is for improving physicians’ clinical practice?
How useful do you believe OSCE training is for avoiding lawsuits?
How useful do you believe the OSCE training is for teaching physicians how to effectively communicate with patients?
How useful do you believe OSCE training is for teaching physicians how to effectively communicate with other providers?
Motive of the ABA with regard to the OSCE introduction
What do you think is the main motive of the ABA to start OSCE? (financial gain to the ABA, improve patient care, increase patient satisfaction, decrease disciplinary proceeding against anesthesiologists by the licensing bodies)
Future of the OSCE
What do you think should be the future of the OSCE? (permanently scrapped, paused pending demonstration that it improves patient outcomes, paused pending demonstration that it decreases disciplinary proceeding against anesthesiologists by the licensing bodies, should continue as is)

OSCE=Objective structured clinical examination, US=Ultrasound, TEE=Transesophageal echocardiogram, ABA=American Board of Anesthesiology

The following provides the anesthesiology programs to whose residents the questionnaire surveys were sent

Programs in the United States to which the questionnaire was E-mailed
University of Alabama Medical Center Program
UAMS College of Medicine Program
Little Rock, AR
Mayo Clinic College of Medicine and Science (Arizona) Program
University of Arizona College of Medicine-Tucson Program
Loma Linda University Health Education Consortium Program
University of Southern California/LAC+USC Medical Center Program
Cedars-Sinai Medical Center Program
UCLA David Geffen School of Medicine/UCLA Medical Center Program
Riverside University Health System Program
University of California (Irvine) Program
Stanford Combined Pediatrics Anesthesiology Residency
Stanford Health Care-Sponsored Stanford university Program
University of California Davis Health Program
Naval Medical Center (San Diego) Program
University of California (San Diego) Medical Center Program
University of California (San Francisco) Program
Los Angeles County-Harbor-UCLA Medical Center Program
KDHCD Program
University of Colorado Program
University of Connecticut Program
Yale-New Haven Medical Center Program
MedStar health/Georgetown University Hospital Program
George Washington University Program
HCA Healthcare/USF Morsani College of Medicine GME/Oak Hill Hospital Program
University of Florida Program
University of Florida College of Medicine Jacksonville Program
Mayo clinic college of Medicine and Science (Jacksonville) Program
HCA healthcare East Florida division GME/Kendall Regional Medical Center Program
University of Miami/Jackson Health System Program
Mount Sinai Medical Center of Florida Inc Program
University of Central Florida/HCA Healthcare GME (Ocala) Program
Memorial Healthcare System (Hollywood Florida) Program
Larkin Community Hospital Program
Cleveland Clinic (Florida) Program
Emory University School of Medicine Program
Medical college of Georgia Program
University of Iowa Hospitals and Clinics Program
University of Illinois College of Medicine at Chicago Program
University of Chicago Program
Cook County Health and Hospitals System Program
Advocate Health Care (Advocate Illinois Masonic Medical Center) Program
McGaw Medical Center of Northwestern University Program
Rush University Medical Center Program
Loyola University Medical Center Program
Indiana University School of Medicine Program, Indianapolis, In
University of Kansas School of Medicine Program
University of Kansas (Wichita) Program
University of Kentucky College of Medicine Program
University of Louisville School of Medicine Program
Tulane University Program
Louisiana State University Program
Ochsner Clinic Foundation Program
Louisiana State University (Shreveport) Program
Beth Israel Deaconess Medical Center Program
Boston University Medical Center Program
Tufts Medical Center Program
Brigham and Women’s Hospital Program
St Elizabeth’s Medical Center Program
Massachusetts General Hospital Program
Children’s Hospital Program
Lahey Clinic Program
UMMS-Baystate Program
University of Massachusetts Program
University of Maryland Program
Johns Hopkins University Program
National Capital Consortium Program
Maine Medical Center Program
University of Michigan Health System Program
Henry Ford Hospital/Wayne State University Program
Detroit Medical Center/Wayne State University Program
McLaren Health Care/Greater Lansing/Msu Program
Wayne State University School of Medicine Program
McLaren Health Care/Oakland/Msu Program
Beaumont Health (Royal Oak) Program
University of Minnesota Program
Mayo Clinic College of Medicine and Science (Rochester) Program
University of Missouri-Columbia Program
University of Missouri-Kansas City School of Medicine Program
Still OPTI/Northeast Regional Medical Center Program
Washington University/B-JH/SLCH Consortium Program
St Louis University School of Medicine Program
University of Mississippi Medical Center Program
University of North Carolina Hospitals Program
Duke University Hospital Program
Wake Forest University School of Medicine Program
University of Nebraska Medical Center College of Medicine Program
Dartmouth-Hitchcock/Mary Hitchcock Memorial Hospital Program
Cooper medical School of Rowan University/Cooper University Hospital Program
Hackensack University Medical Center Program
Rutgers health/St Barnabas Medical Center Program Hackensack, NJ
Rutgers health/Robert Wood Johnson Medical School Program, New Brunswick, NJ
St Joseph’s University Medical Center Program
University of New Mexico School of Medicine Program
HCA Healthcare Sunrise Health Graduate Medical Education Program
Albany Medical Center Program
Montefiore Medical Center/Albert Einstein College of Medicine Program
Maimonides Medical Center Program
New York-Presbyterian Brooklyn Methodist Hospital Program
SUNY Health Science Center at Brooklyn Program
University at Buffalo Program
Zucker School of Medicine at Hofstra/Northwell Program
New York Medical College at Metropolitan Hospital Program
New York Presbyterian Hospital (Columbia Campus) Program
Icahn School of Medicine at Mount Sinai (Morngingside/West) Program
New York Presbyterian Hospital (Cornell Campus) Program
Icahn School of Medicine at Mount Sinai (Mount Sinai Hospital) Program
Health Quest Program
University of Rochester Program
Stony Brook Medicine/University Hospital Program
SUNY Upstate Medical University Program
Westchester Medical Center Program
University of Cincinnati Medical Center/College of Medicine Program
Case Western Reserve University/University Hospitals Cleveland Medical Center Program
The Metro Health System/Case Western Reserve University Program
Cleveland Clinic Foundation Program
OhioHealth/Doctors Hospital Program
Ohio State University Hospital Program
Kettering Health Network Program
University of Toledo Program
Cleveland Clinic Foundation/South Pointe Hospital Program
University of Oklahoma Health Sciences Center Program
Oklahoma State University Center for Health Sciences Program
Oregon Health and Science University Program
Geisinger Health System Program
Penn State Milton S Hershey Medical Center Program
UPMC Lititz Program
Sidney Kimmel Medical College at Thomas Jefferson University/TJUH Program
Temple University Hospital Program
UPMC Medical Education Program
Allegheny Health Network Medical Education Consortium (AGH/WPH) Program
Robert Packer Hospital Program
Tower Health Program
University of Puerto Rico Program
Brown University/Rhode Island Hospital-Lifespan Program
Medical University of South Carolina Program
HCA Healthcare Grand Strand Regional Medical Center Program
University of Tennessee Medical Center at Knoxville Program
Vanderbilt University Medical Center Program
University of Texas Southwestern Medical Center Program
University of Texas Medical Branch Hospitals Program
Baylor College of Medicine Program
University of Texas Health Science Center at HOUSTON PROGRAM
San Antonio Uniformed Services Health Education Consortium Program
Texas Tech University Health Sciences Center at LUBBOCK PROGRAM
University of Texas Health Science center San Antonio Joe and Teresa Lozano Long School of Medicine Program
Texas A and M College of Medicine-Scott and White Medical Center (Temple) Program
University of Utah Health Program
University of Virginia Medical Center Program
Naval Medical Center (Portsmouth) Program
Virginia Commonwealth University Health System Program
University of Vermont Medical Center Program
Virginia Mason Medical Center Program
University of Washington Program
University of Wisconsin Hospitals and Clinics Program
Medical College of Wisconsin Affiliated Hospitals Program
West Virginia University Program

UAMS=University of Arkansas for Medical Sciences, KDHCD=Kaweah Delta Health Care District, TJUH=Thomas Jefferson University, LAC=Los Angeles County, USC=University of Southern California, UPMC= University of Pittsburgh Medical Center, AGH=Allegheny General Hospital, WPH=White Plains Hospital, USF=University of South Florida, HCA=Hospital Corporation of America, GME=Graduate Medical Education