Academic performance of ethnic minority candidates and discrimination in the MRCGP examinations between 2010 and 2012: analysis of data.

OBJECTIVE: To determine the difference in failure rates in the postgraduate examination of the Royal College of General Practitioners (MRCGP) by ethnic or national background, and to identify factors associated with pass rates in the clinical skills assessment component of the examination.
DESIGN: Analysis of data provided by the Royal College of General Practitioners and the General Medical Council. PARTICIPANTS: Cohort of 5095 candidates sitting the applied knowledge test and clinical skills assessment components of the MRCGP examination between November 2010 and November 2012. A further analysis was carried out on 1175 candidates not trained in the United Kingdom, who sat an English language capability test (IELTS) and the Professional and Linguistic Assessment Board (PLAB) examination (as required for full medical registration), controlling for scores on these examinations and relating them to pass rates of the clinical skills assessment.
SETTING: United Kingdom.
RESULTS: After controlling for age, sex, and performance in the applied knowledge test, significant differences persisted between white UK graduates and other candidate groups. Black and minority ethnic graduates trained in the UK were more likely to fail the clinical skills assessment at their first attempt than their white UK colleagues (odds ratio 3.536 (95% confidence interval 2.701 to 4.629), P<0.001; failure rate 17% v 4.5%). Black and minority ethnic candidates who trained abroad were also more likely to fail the clinical skills assessment than white UK candidates (14.741 (11.397 to 19.065), P<0.001; 65% v 4.5%). For candidates not trained in the UK, black or minority ethnic candidates were more likely to fail than white candidates, but this difference was no longer significant after controlling for scores in the applied knowledge test, IELTS, and PLAB examinations (adjusted odds ratio 1.580 (95% confidence interval 0.878 to 2.845), P=0.127).
CONCLUSIONS: Subjective bias due to racial discrimination in the clinical skills assessment may be a cause of failure for UK trained candidates and international medical graduates. The difference between British black and minority ethnic candidates and British white candidates in the pass rates of the clinical skills assessment, despite controlling for prior attainment, suggests that subjective bias could also be a factor. Changes to the clinical skills assessment could improve the perception of the examination as being biased against black and minority ethnic candidates. The difference in training experience and other cultural factors between candidates trained in the UK and abroad could affect outcomes. Consideration should be given to strengthening postgraduate training for international medical graduates.

Introduction

In order to practise as an accredited general practitioner (family physician) in the United Kingdom, doctors must pass the MRCGP (Membership of the Royal College of General Practitioners) examination, which is set by the Royal College of General Practitioners. It has been known that there is a difference in pass rates in the MRCGP examination for international medical graduates when compared with UK graduates, and British black and minority ethnic graduates when compared with white UK graduates.1 Differences in pass rates in postgraduate medical examinations are not restricted to those set by the Royal College of General Practitioners. The Royal College of Psychiatrists2 and Royal College of Physicians3 have commissioned and published data highlighting the increased failure rate of international medical graduates and British black and minority ethnic doctors in their postgraduate examinations.

Differences in pass rates between native and international medical graduates have also been highlighted in postgraduate examinations in Australia and the United States.4 5 With continuing dependence on international medical graduates in meeting the workforce needs of many developed countries, understanding the barriers that this group of doctors face in entering and completing specialist medical training will be important. For example, in 2012, non-UK qualifiers accounted for almost a quarter of the general practitioner workforce in the UK,6 and black and minority ethnic doctors trained in the UK constitute nearly one third of graduating doctors (unpublished data). This study aimed to understand the reasons for the differences in pass rates, using previously unavailable data provided by the General Medical Council and the Royal College of General Practitioners. We were able to analyse pass rates in the MRCGP examination controlling for age, sex, prior attainment in multiple choice examinations and in examinations that are part of the registration requirements for international medical graduates.

General practice specialty training

Entry into specialty training for general practice is now organised centrally through the National Recruitment Office for General Practice Training. Training usually takes three years after the completion of the two years of foundation year training. In 40% of foundation programmes in England, the training will include four months working in a general practice environment. The three year training will usually consist of working in a general practice for 18 months under the supervision of a trainer, and in educationally approved hospital posts relevant to the work of a general practitioner for the remaining time. While in general practice, the trainee will follow the approved curriculum—learning how general practice is organised and managed—and will see patients both in the surgery and in their homes.

Pathways to enter general practice for international medical graduates—with the exception of those who have graduated from the European Economic Area—are likely to be different. Although some international medical graduates complete foundation training posts, most will come through the full registration route regulated by the General Medical Council. To be considered for full registration, international medical graduates need to complete an English language capability test (IELTS) by scoring a minimum of seven points in all components (speaking, reading, writing, and listening) of the academic version of IELTS.

They must also sit the Professional and Linguistic Assessment Board (PLAB) examination, which consists of two parts. The second part is an objective structured clinical examination, similar in some aspects to the clinical skills assessment section of the MRCGP examination (see below). It assesses the competencies of international medical graduates to practise medicine safely in UK hospitals, and the standard is set at the level of what would be expected of a trainee completing foundation year 1. The components of the test assess clinical examination, practical skills, communication skills, and history taking. Once international medical graduates have achieved foundation competences (or have demonstrated equivalence), they are eligible to apply for general practice specialist training through the National Recruitment Office. Some (but not all) candidates from the European Economic Area do take the IELTS and PLAB part 2, and these individuals have formed a useful comparison group for this study. We analysed pass rates in the MRCGP examination, controlling for prior performance in the IELTS and PLAB part 2 examinations.

At the end of training, satisfactory completion of the MRCGP examination is a prerequisite for practising as a general practitioner. The concerns regarding the failure rate of international medical graduates and British black and minority ethnic graduates are therefore also important, since failure to pass the MRCGP examination effectively means that a person cannot work in general practice in any capacity. The number of attempts at each component of the MRCGP examination is now restricted to four attempts.

The MRCGP examination comprises three separate components: an applied knowledge test, a clinical skills assessment, and a workplace based assessment. Each component tests different competences using validated assessment methods; together, they cover the spectrum of knowledge, skills, behaviours, and attitudes defined by the curriculum for general practice specialty training.

The applied knowledge test is a summative assessment of the knowledge base that underpins independent general practice in the UK. It is a machine marked, multiple choice examination, and can be sat during or after the second year of training. The clinical skills assessment is a summative assessment of a doctor’s ability to integrate and apply clinical, professional, communication, and practical skills appropriate for general practice. The format of the examination simulates a typical surgery clinic in the UK’s health service and assesses a range of scenarios from general practice. It can be sat during or after the third year of training. The workplace based assessment, defined by the curriculum, evaluates the trainee’s progress in areas of professional practice best tested in the workplace, and is a continuous and formative assessment carried out by a designated GP. It is overseen by the postgraduate deanery—which is the organisation responsible for postgraduate training of doctors in the UK after qualification. These deaneries are organised geographically.

Before 2010, the clinical skills assessment involved the candidate undertaking 13 clinical scenarios (cases); although all 13 cases were marked, only 12 cases were counted towards the candidate’s overall score. The 13th case was used to pilot new cases and did not contribute to the candidate’s overall mark. The passing standard for the assessment was based on a “number of cases to pass” methodology. Under this methodology, eight marginal passes with four clear fails was a pass, whereas seven clear passes with five marginal fails was an overall fail. The Royal College of General Practitioners and General Medical Council were concerned that this method of standard setting could allow doctors to pass who were not safe. They therefore introduced a change in the standard setting method that took account of the pass-fail borderline. This change improved the reliability of the assessment and allowed the examiners to compensate between cases and domains in setting the standard.

In normal circumstances, the examiners would mark against domains with one examiner marking each candidate they observe on a case by giving them one of four grades. Each candidate is also graded against three domains: data gathering, technical and assessment skills, clinical management skills, and interpersonal skills. With the borderline group methodology, examiners make a further standard setting judgment, rating the candidate as pass, borderline, or fail. For each case, the marks of those candidates marked as borderline are averaged. These averaged borderline scores are then aggregated across all 13 cases to create the “cut score”—that is, the approximation between a passing and a failing score. The final, actual pass mark has an adjustment to the overall cut score to take account of the measurement error inherent in any assessment process of this kind.

This borderline method of marking examinations is widely used both internationally and in the UK. It is the standard method in some medical schools when assessing students in clinical examinations (widely known as objective structured clinical examinations). It is also used with other marking schemes by the General Medical Council in marking the PLAB part 2 examination. The Royal College of General Practitioners adopted the borderline group methodology in the MRCGP examination in 2010, and also included the 13th clinical scenario as a marked case.

Methods

Our aim was to identify the factors associated with pass rates in the clinical skills assessment, controlling for age, sex, performance in the applied knowledge test, IELTS, and PLAB part 2 examination. We focused on the clinical skills assessment because of concerns raised by the General Medical Council as to the possibility of bias against certain groups, including men, British candidates of ethnic minority, and international medical graduates. We did not have data on other prematriculation covariates (for example, score on entry and exit from medical school). Data on the cohort of candidates who did the clinical skills assessment between November 2010 (when the important changes were introduced to the method of marking of the assessment) and November 2012 were analysed. We were given outcome data on the applied knowledge test and demographic data that included region of primary medical qualification and ethnicity on 5094 candidates (of a total of 5744).

Ethnicity information was self reported and 2484 candidates described themselves as English, Welsh, Scottish, or Northern Irish; Irish; or “white other.” Categories for black and minority ethnic candidates were derived from the UK census classification and included black Caribbean, black African, Indian, Pakistani, Bangladeshi, black other, Asian other, Chinese, Arab, mixed heritage groups, and “other ethnic group.” A total of 1160 candidates were assigned to the BME group. Candidates who did not provide this information (n=489) or who had data missing (n=61) were excluded from the analysis. Additionally, we were provided with data on candidates who took the PLAB part 2 (n=1207) and IELTS examinations (n=1175), from a potential total of 1310 candidates. These data included IELTS scores and scores for individual components of the PLAB examination. We did not use outcome data from the PLAB part 1 examination, because we already had data from the applied knowledge test, which is a similar machine marked test. The applied knowledge test has the advantage of being taken shortly before candidates attempt the clinical skills assessment. In our view, the applied knowledge test was likely to be a better predictor of the clinical skills assessment than the PLAB part 1 examination.

For the purpose of our analyses, we assumed that the score on the applied knowledge test provided a measure of performance that was not influenced by possible subjective biases regarding region of primary medical qualification or ethnicity. We used the score for the first attempt at the applied knowledge test and estimated, using logistic regression, the odds ratio for failure at the clinical skills assessment. We estimated odds ratios by comparing every other combination of region of primary medical qualification and ethnicity against white UK candidates adjusted for age, sex, and scores on the applied knowledge test. Because the pass mark for the test varies between sittings, this was also included as a covariate. If age, sex, or clinical knowledge (as measured by the applied knowledge test) correlate with region of primary medical qualification or ethnicity, any difference in outcome could be related to these factors. It is therefore appropriate to obtain estimates of the difference in clinical skills assessment performance adjusted for these factors as well as obtain unadjusted estimates. Because the failure rate might vary between deaneries, we also included a random effect for the postgraduate deanery of the general practice candidates in the model to prevent overprecision of estimates—that is, avoiding confidence intervals that were overly narrow. The intracluster correlation has been given as a measure of clustering by deanery, with a value equal to zero implying no clustering effect.7

Results

Table 1 shows the difference in pass rates in the clinical skills assessment and applied knowledge test by region of primary medical qualification and age and sex. There were substantial differences in pass rates between candidates who were black or minority ethnic and those who were international medical graduates when compared with white UK graduates. Compared with white UK graduates, all the other five groups defined by ethnicity and region of primary medical qualification did significantly worse at their first attempt. The smallest difference was with British black and minority ethnic candidates (odds ratio 4.776 (95% confidence interval 3.709 to 6.148), P<0.001) and the greatest for black and minority ethnic candidates from the European Economic Area (45.732 (23.938 to 87.368), P<0.001).

Table 1

Age, sex, and failure rates of candidates at clinical skills assessment and applied knowledge test and scores of international medical graduates in IELTS and PLAB examinations

	Ethnicity and region of primary medical qualification
	White IMG	BME IMG	White EEA	BME EEA	White UK	BME UK
Age (years) at first attempt of clinical skills assessment	36.97 (4.86)	36.39 (4.66)	34.29 (3.91)	35.31 (4.65)	30.47 (3.83)	30.06 (2.90)
Male sex (No (%))	32 (37.2)	713 (58.3)	33 (35.5)	29 (60.4)	791 (31.9)	513 (44.2)
Clinical skills assessment (first attempt)
Fail (No (%); % of men, women)	41 (47.7; 62.5, 38.9)	798 (65.2; 74.6, 52.1)	30 (32.3; 39.4, 28.3)	33 (68.8; 75.9, 57.9)	110 (4.4; 6.8, 3.4)	198 (17.1; 25.7, 10.2)
Data gathering, technical and assessment skills	26.53 (4.19)	24.89 (3.58)	26.96 (3.60)	23.94 (4.67)	30.44 (3.28)	29.02 (3.59)
Clinical management skills	23.19 (4.70)	21.26 (4.12)	24.03 (4.00)	20.79 (4.84)	27.95 (3.56)	25.59 (4.02)
Interpersonal skills	24.67 (5.56)	22.31 (4.45)	25.82 (4.25)	22.50 (5.63)	30.79 (3.47)	28.49 (4.16)
Total score relative to pass mark	0.97 (13.37)	−5.07 (10.91)	3.34 (10.57)	−6.29 (14.13)	15.58 (9.11)	9.47 (10.62)
Total No of candidates	86	1224	93	48	2484	1160
Applied knowledge test (first attempt)
Fail (No (%))	35 (40.7)	517 (42.2)	27 (29.0)	34 (70.8)	226 (9.1)	244 (21.0)
Clinical medicine	70.29 (9.00)	70.58 (9.11)	73.22 (9.14)	62.45 (10.77)	78.59 (7.85)	75.00 (8.60)
Evidence interpretation	63.47 (17.15)	61.66 (15.34)	70.79 (14.44)	56.67 (15.45)	79.47 (12.38)	74.94 (13.03)
Organisational questions	65.70 (13.22)	63.77 (12.75)	70.25 (12.85)	58.65 (14.65)	77.24 (11.07)	71.55 (11.69)
Total score relative to pass mark	1.67 (18.16)	1.72 (17.38)	8.94 (17.40)	−13.75 (19.41)	20.67 (14.81)	12.77 (16.18)
Total No of candidates	86	1224	93	48	2483	1160
IELTs
Reading	7.33 (0.67)	7.23 (0.76)	7.55 (1.06)	7.04 (0.63)	—	—
Speaking	7.52 (0.70)	7.51 (0.65)	7.82 (0.75)	8.15 (0.90)	—	—
Understanding	7.48 (0.70)	7.49 (0.75)	7.41 (0.92)	7.73 (0.93)	—	—
Writing	6.78 (0.69)	7.15 (0.68)	7.27 (0.79)	6.88 (1.00)	—	—
Overall	7.35 (0.52)	7.42 (0.46)	7.55 (0.65)	7.54 (0.56)	—	—
Total No of candidates	60	1091	11	13	—	—
PLAB part 2
Communication	3.34 (0.50)	3.14 (0.47)	3.45 (0.42)	3.13 (0.58)	—	—
Examination	3.28 (0.47)	3.18 (0.48)	3.36 (0.64)	3.04 (0.48)	—	—
History	3.28 (0.50)	3.17 (0.43)	3.46 (0.44)	2.96 (0.33)	—	—
Practice	3.34 (0.65)	3.41 (0.59)	3.68 (0.42)	3.25 (0.57)	—	—
Total No of candidates	64	1117	11	15	—	—

Data are mean (standard deviation) unless stated otherwise. IMG=international medical graduate; BME=black and minority ethnic; EEA=European Economic Area.

Table 2 shows the distribution of candidate characteristics by deanery. We observed a large variation in the proportion of candidates who were black and minority ethnic and those who were international medical graduates. Some deaneries had a much lower proportion of white UK candidates than other candidate groups.

Table 2

Distribution of general practice trainees in deaneries, by ethnicity and region of primary medical qualification

	Ethnicity and region of primary medical qualification						Total No of trainees
	White UK	BME UK	White IMG	BME IMG	White EEA	BME EEA
Defence	36 (94.7)	2 (5.3)	0	0	0	0	38
East Midlands	111 (38.0)	64 (21.9)	6 (2.1)	104 (35.6)	4 (1.4)	3 (1.0)	292
East of England	122 (31.8)	83 (21.6)	7 (1.8)	157 (40.9)	9 (2.3)	6 (1.6)	384
Kent, Surrey, and Sussex	145 (29.8)	117 (24.0)	16 (3.3)	184 (37.8)	16 (3.3)	9 (1.8)	487
London	267 (42.9)	298 (47.9)	10 (1.6)	39 (6.3)	4 (0.6)	4 (0.6)	622
Mersey	116 (53.2)	29 (13.3)	3 (1.4)	61 (28.0)	5 (2.3)	4 (1.8)	218
Scotland (East)	32 (72.7)	4 (9.1)	2 (4.5)	5 (11.4)	1 (2.3)	0	44
Scotland (North)	44 (67.7)	4 (6.2)	4 (6.2)	12 (18.5)	1 (1.5)	0	65
Scotland (South East)	63 (71.6)	11 (12.5)	0 (0.0)	12 (13.6)	2 (2.3)	0	88
Scotland (West)	137 (70.6)	20 (10.3)	2 (1.0)	32 (16.5)	3 (1.5)	0	194
Oxford	106 (59.9)	51 (28.8)	1 (0.6)	17 (9.6)	2 (1.1)	0	177
Wessex	114 (61.6)	29 (15.7)	8 (4.3)	27 (14.6)	4 (2.2)	3 (1.6)	185
Peninsula	85 (80.2)	5 (4.7)	4 (3.8)	9 (8.5)	3 (2.8)	0	106
Severn	155 (77.5)	20 (10.0)	3 (1.5)	17 (8.5)	4 (2.0)	1 (0.5)	200
West Midlands	178 (32.4)	163 (29.7)	6 (1.1)	187 (34.1)	7 (1.3)	8 (1.5)	549
North Western	159 (39.6)	115 (28.6)	5 (1.2)	113 (28.1)	5 (1.2)	5 (1.2)	402
Northern	137 (59.6)	24 (10.4)	4 (1.7)	53 (23.0)	10 (4.3)	2 (0.9)	230
Northern Ireland	95 (93.1)	1 (1.0)	0	1 (1.0)	5 (4.9)	0	102
Wales	116 (60.7)	22 (11.5)	1 (0.5)	48 (25.1)	3 (1.6)	1 (0.5)	191
Yorkshire and Humber	238 (54.1)	73 (16.6)	4 (0.9)	118 (26.8)	5 (1.1)	2 (0.5)	440
Not specified	28 (34.6)	25 (30.9)	0	28 (34.6)	0	0	81
Total No of trainees	2484 (48.8)	1160 (22.8)	86 (1.7)	1224 (24.0)	93 (1.8)	48 (0.9)	5095

Data are number (%) of trainees unless stated otherwise. IMG=international medical graduate; BME=black and minority ethnic; EEA=European Economic Area.

Table 3 shows the unadjusted and adjusted odds ratios for failure in the clinical skills assessment, adjusted for sex, age, and score on the applied knowledge test. Women and candidates with higher scores on their applied knowledge test were less likely to fail the clinical skills assessment at the first attempt. Older candidates were also less likely to pass. All five candidate groups had a significantly higher rate of failure than white UK graduates (adjusted odds ratio 3.536 (95% confidence interval 2.701 to 4.629), P<0.001 for black and minority ethnic applicants who trained in the UK; 14.741 (11.397 to 19.065), P<0.001 for black and minority ethnic candidates who were international medical graduates). There was only slight evidence of clustering, with an intracluster correlation of 0.0246 in the unadjusted model, reducing to 0.009 in the adjusted model). The reduction in the intracluster correlation was expected, because a covariate will tend to remove difference in the characteristic of candidates between deaneries. The differences between candidate groups tended to decrease with subsequent sittings of the clinical skills assessment, disappearing for British black and minority ethnic graduates at the second attempt (although this finding was not significant). However, differences persisted for black and minority ethnic candidates who were international medical graduates at the second and third attempts.

Table 3

Logistic regression models of failure rates of clinical skills assessment (unadjusted and adjusted for sex, age, and scores for applied knowledge test scores for first three attempts)

Candidate group	First attempt*			Second attempt†			Third attempt‡
	Odds ratio (95% CI)	P		Odds ratio (95% CI)	P		Odds ratio (95% CI)	P
Unadjusted model
BME UK	4.776 (3.709 to 6.148)	<0.001		1.086 (0.600 to 1.966)	0.786		3.876 (0.941 to 15.965)	0.061
White IMG	19.432 (12.134 to 31.117)	<0.001		5.798 (2.641 to 12.731)	<0.001		6.373 (1.348 to 30.129)	0.019
BME IMG	39.080 (31.022 to 49.232)	<0.001		6.735 (4.101 to 11.059)	<0.001		11.769 (3.394 to 40.810)	<0.001
White EEA	9.753 (6.037 to 15.756)	<0.001		3.673 (1.485 to 9.086)	0.005		3.237 (0.570 to 18.378)	0.185
BME EEA	45.732 (23.938 to 87.368)	<0.001		6.786 (2.858 to 16.113)	<0.001		12.462 (2.467 to 62.954)	0.002
Model constant	0.047 (0.037 to 0.059)	<0.001		0.233 (0.145 to 0.375)	<0.001		0.177 (0.052 to 0.602)	0.006
ICC	0.0246, AIC= 3837.9	—		<0.001, AIC= 1501.3	—		<0.001, AIC=683.0	—
Adjusted model
BME UK	3.536 (2.701 to 4.629)	<0.001		1.013 (0.537 to 1.912)	0.968		5.080 (1.144 to 22.566)	0.033
White IMG	7.171 (4.246 to 12.110)	<0.001		3.693 (1.593 to 8.563)	0.002		7.643 (1.492 to 39.156)	0.015
BME IMG	14.741 (11.397 to 19.065)	<0.001		4.380 (2.561 to 7.491)	<0.001		11.406 (3.068 to 42.403)	<0.001
White EEA	5.540 (3.296 to 9.313)	<0.001		3.475 (1.331 to 9.069)	0.011		4.663 (0.760 to 28.621)	0.096
BME EEA	10.144 (5.040 to 20.419)	<0.001		2.858 (1.143 to 7.143)	0.025		10.888 (1.988 to 59.615)	0.006
Female sex	0.446 (0.374 to 0.532)	<0.001		0.433 (0.329 to 0.570)	<0.001		0.503 (0.328 to 0.770)	0.002
Age (in years) at examination	1.088 (1.066 to 1.111)	<0.001		1.063 (1.033 to 1.093)	<0.001		1.087 (1.041 to 1.136)	<0.001
Scores for applied knowledge test
Clinical medicine	0.990 (0.984 to 0.997)	0.006		0.995 (0.986 to 1.005)	0.348		1.001 (0.986 to 1.016)	0.898
Evidence interpretation	0.986 (0.978 to 0.994)	0.001		0.976 (0.964 to 0.987)	<0.001		1.004 (0.987 to 1.021)	0.659
Organisational questions	1.028 (0.994 to 1.063)	0.114		1.002 (0.956 to 1.050)	0.937		0.980 (0.925 to 1.037)	0.486
Pass mark§	0.048 (0.000 to 4.854)	0.197		2.789 (0.005 to 1665.8)	0.753		2.481 (0.001 to 6226.7)	0.82
Model constant	0.990 (0.984 to 0.997)	0.006		0.995 (0.986 to 1.005)	0.348		1.001 (0.986 to 1.016)	0.898
ICC	0.009, AIC=3402.8	—		<0.001, AIC=1378.9	—		<0.001, AIC=652.8	—

BME=black and minority ethnic; IMG=international medical graduate; EEA=European Economic Area; ICC=intracluster correlation coefficient; AIC=Akaike information criterion; model constant=constant of the logistic regression model.

*Unadjusted, n=5095; adjusted, n=5094.

†Unadjusted, n=1188; adjusted, n=1188.

‡Unadjusted, n=533; adjusted, n=532.

§The pass mark of the applied knowledge test varies between sittings.

A logistic regression model was also fitted with a random effect for deanery to investigate the association between outcomes of the clinical skills assessment and the applied knowledge test, IELTS, and PLAB part 2 examination. This model was based on data for non-UK candidates; the comparisons were between black and minority ethnic and white candidates and between European Economic Area and international medical graduates. Details of the covariates are the same as for the model in table 3 with the addition of covariates for IELTS components and PLAB part 2. Table 4 gives the unadjusted and adjusted analysis of failure rates in the clinical skills assessment, controlling for the applied knowledge test, PLAB part 2, and IELTS component scores. In this cohort of non-UK candidates, black and minority ethnic candidates were more likely to fail than white candidates, but this was no longer significant (adjusted odds ratio 1.580 (95% confidence interval 0.878 to 2.845, P=0.127). Furthermore, the odds ratio was closer to 1 than the corresponding adjusted odds ratio (odds ratio 3.764), which can be derived from the analysis from table 3 without adjustment for IELTS and PLAB part 2 outcomes.

Table 4

Logistic regression models for failure at first attempt at clinical skills assessment for non-UK graduates, adjusted for age, sex, and performance in applied knowledge test, IELTS, and PLAB part 2 examinations

	Odds ratio (95% CI)	P
Unadjusted model (n=1166)
Black and minority ethnic v white candidates	2.533 (1.731 to 3.706)	<0.001
European Economic Area v international medical graduates	0.776 (0.508 to 1.184)	0.239
Model constant	0.741 (0.506 to 1.084)	0.123
Intracluster correlation for deanery	0.010	—
Adjusted model (n=1166)
Black and minority ethnic v white candidates	1.580 (0.878 to 2.845)	0.127
European Economic Area v international medical graduates	0.968 (0.352 to 2.660)	0.95
Female sex	0.497 (0.377 to 0.655)	<0.001
Age (in years) at time of clinical skills assessment	1.101 (1.062 to 1.142)	<0.001
Applied knowledge test
Clinical medicine	0.963 (0.945 to 0.981)	<0.001
Evidence interpretation	0.996 (0.986 to 1.007)	0.487
Organisational questions	0.990 (0.977 to 1.002)	0.11
Mark to pass	1.016 (0.967 to 1.068)	0.522
IELTS
Reading score	0.900 (0.731 to 1.108)	0.321
Speaking score	0.842 (0.673 to 1.054)	0.133
Understanding score	0.719 (0.590 to 0.877)	0.001
Writing score	1.004 (0.814 to 1.239)	0.968
PLAB part 2
Communication	0.606 (0.440 to 0.836)	0.002
Examination	0.880 (0.654 to 1.185)	0.4
History	0.719 (0.514 to 1.007)	0.055
Practical	0.871 (0.690 to 1.098)	0.241
Model constant	632.191 (0.465 to 862 069)	0.08
Intracluster correlation for deanery	<0.001	—

Model constant=constant of the logistic regression model.

These analyses show that issues related to what we term “clinical understanding” are probably related to and predictive of failure rates in the clinical skills assessment. Clinical understanding is a complicated concept that could include linguistic understanding as well as clinical reasoning and communication, and could be indirectly measured by components of the IELTS, PLAB part 2, and applied knowledge test. We were not able to directly compare UK and non-UK graduates regarding this term, because UK graduates do not sit the IELTS and PLAB examinations. However, for non-UK graduates, the odds ratio of failure rates between black and minority ethnic and white candidates fell from 3.764 (95% confidence interval 1.451 to 9.77, P=0.006) to 1.580 (0.878 to 2.845, P=0.127) when these components were taken into account. This reduction suggests that taking into account the outcomes of IELTS, PLAB part 2, and applied knowledge test could be important in understanding why international medical graduates have such a high failure rate in the clinical skills assessment.

Discussion

Our results showed significant differences in outcome in the applied knowledge test and clinical skills assessment components of the MRCGP examinations, between black and minority ethnic candidates (trained in the UK or abroad) and white candidates trained in the UK. We cannot exclude subjective bias owing to racial discrimination in the marking of the clinical skills assessment as a reason for these differential outcomes. Previous training experience and cultural factors (which include physician-patient relationships, and communication and proficiency in spoken English) could help explain these differences between UK candidates and international medical graduates. However, these cultural factors cannot explain differences between white candidates and black and minority ethnic candidates who have trained in the UK, and who would have had similar training experiences and language proficiency.

The applied knowledge test is a machine marked examination that tests applied clinical knowledge. The pass rates in this test for both British black and minority ethnic graduates and international medical graduates differed from those for white UK graduates. It is difficult to attribute this to bias because it is a machine marked test. For UK graduates, differences between white candidates and black and minority ethnic candidates may seem to reflect existing observed differences in examination performance, which have described both in higher education and in medical examinations.8 9 There is a general consensus that the reasons for this difference are complex and cannot be explained by factors such as prior attainment, social class, and school background, and this is supported by our analyses.

The differential outcomes decreased in subsequent sittings of the clinical skills assessment, especially between the first and second sittings. This finding could be because candidates who should have passed the first time but failed to do so because of bias in the examination, are passing the exam at subsequent attempts. In effect, there is a selection bias in subsequent cohorts because of conditioning on the previous result. This effect makes interpreting the results of the second and third attempts more difficult to interpret, which is why we focused on the first attempt at the clinical skills assessment to draw our conclusions.

Implications of results for clinical examinations

The biggest difference in failure rates was between graduates trained in the UK and those who trained abroad (including in the European Economic Area), suggesting that the preparedness of the candidates based on previous education experience could be a factor. The vast majority of international medical graduates come from the Indian subcontinent and from countries where the discipline of general practice is very different. International medical graduates will therefore have much less direct experience of this specialty than their UK counterparts. The clinical skills assessment is not a culturally neutral examination, and the cultural norms of what are expected in a consultation will vary from country to country. It cannot be described simply as a clinical examination testing clinical knowledge, but is designed to ensure that doctors are safe to practice in UK general practice. British graduates have much more exposure and training in general practice both through medical school training and the foundation training programme than most international medical graduates whose home countries might not have health systems as dominated by primary care as the NHS is in the UK. In our view, these factors could disadvantage international medical graduates in subtle ways and explain the much larger differences in outcomes between UK and non-UK graduates.

The clinical skills assessment and its marking is based on a well established pedagogy that is internationally recognised and used widely in postgraduate examinations.10 However, like any clinical examination, it is subject to bias. We cannot ascertain if the standardised patients (played by actors) behaved differently in front of candidates from non-white ethnic groups. Nor can we confidently exclude bias from the examiners in the way that they assessed non-white candidates. There is mandatory training of RCGP examiners in equality and diversity issues, and there is training and monitoring of the actors to ensure consistency in the presentation of the cases. There is also a well developed programme of continuing training and feedback to examiners of their performance. The RCGP itself has been at the forefront of research to understand the biases caused by oral examinations.11 We would suggest looking at the diversity of examiners and actors, the type of cases included in the examination and the feedback given to candidates as areas for possible improvement. Based on our findings, the RCGP should investigate how both standardised patients and examiners of black and minority ethnic origin would score candidate physicians who are racially and ethnically concordant and compare that to how non-concordant standardised patients and examiners score candidates of black and minority ethnic origin.

Implications of results for postgraduate training

The distribution of international medical graduates and UK black and minority ethnic doctors differed substantially between deaneries (table 2). We have no information on the quality of training in these deaneries, but the combination of selection and training placement systems could operate against the interests of the weaker recruits—in this case, international medical graduates. What this means in practice is that those candidates performing least well at the selection process for general practice are assigned to the least popular training placements, thereby encouraging a cycle of educational deprivation.12 The quality of training could therefore mitigate the effect of differing levels of preparedness among international medical graduates. Deaneries with a large proportion of international medical graduates should perhaps explicitly acknowledge that this group might need additional training support and place the candidates in their stronger training practices or make sure that the trainers are given additional support. Primary care in the UK depends on the recruitment of large numbers of international medical graduates, but most will enter general practice training from a different starting point. The current high rate of failure in the clinical skills assessment is clearly unsatisfactory for both the applicants and service delivery, and suggests the need for additional training for international medical graduates to enable their adaptation to the UK healthcare system.

The high failure rate of ethnic minority candidates in the MRCGP (Membership of the Royal College of General Practitioners) examination has been of concern

It has been questioned whether the clinical skills assessment component of the MRCGP examination discriminates against these individuals

Crude failure rates suggest that nearly 65% of international medical graduates and 17% of ethnic minority graduates trained in the United Kingdom fail the clinical skills assessment at the first attempt

Based on MRCGP examination data from 2010-12, black and minority ethnic graduates trained in the UK were more likely to fail the clinical skills test than their white UK colleagues, despite controlling for age, sex, and scores on the applied knowledge test

Black and minority ethnic graduates trained abroad were substantially more than likely to fail this exam than their white UK colleagues

For non-UK trained candidates, black and minority ethnic candidates were more likely to fail than white candidates, but this difference was no longer significant after controlling for scores in the applied knowledge test, English language capability test (IELTS), and Professional and Linguistic Assessment Board (PLAB) examinations