Could the UK Foundation Programme training post allocation process result in regional variations in the knowledge and skills of Foundation doctors? A cross-sectional study.

BACKGROUND AND AIMS: The allocation of medical school graduates to Foundation Schools (post-qualification training, organized at regional level) in the United Kingdom uses a ranking process that takes into account educational performance at medical school and performance on a situational judgment test (SJT). We aimed to compare the performance of United Kingdom graduates allocated to different United Kingdom Foundation School according to three metrics: educational performance measure (EPM), SJT, and prescribing safety assessment (PSA).
METHODS: We used a cross-sectional study design using data from the UK Medical Education Database, studying 19 United Kingdom Foundation School groups. A total of 33 730 graduates from United Kingdom medical schools in the period 2014 to 2018 (inclusive) who started Foundation Training in August 2018 or earlier were included in the study, excluding those allocated to the Academic Foundation Programme or the Armed Forces Deanery. The outcomes were within-year standardized mean scores (by Foundation School) on the EPM, SJT, and PSA.
RESULTS: There was a significant difference between Foundation Schools in the Educational Performance Measure (F = 401, P < .001), SJT (F = 213, P < .001), and PSA (F = 95, P < .001). Tukey-Kramer pairwise comparisons between Foundation Schools showed a very high percentage of statistical significance (78%, 402/513 comparisons). The Cohen's d effect size for the difference in means and Tukey-Kramer 95% confidence intervals between the Foundation Schools with the highest (North West Thames) and lowest (West Midlands North) means were 1.92 (1.77-2.07) for the EPM, 1.59 (1.44-1.73) for the SJT, and 0.94 (0.79-1.09) for the PSA.
CONCLUSION: There is a statistically significant difference between the knowledge and skills of doctors (as measured by the three metrics used in this study) entering the Foundation Programme in different Foundation Schools. It is less clear whether this has an impact on patient care and thus is unfair from the perspective of the patient.

INTRODUCTION

United Kingdom health legislation is clear in the need to provide an equitable service which reduces, not exacerbates, healthcare inequalities. Patients should, therefore, expect to receive the same quality of care regardless of where in the country they live. One determinant of the quality of care received is the relative performance of the health care professionals providing it, as evidenced in high‐level reviews of serious deficits in health care in the United Kingdom , , , which identified the contribution that staff members make to patient outcomes. Thus if there are differences in professional performance in different settings, then there are also likely to be differences in patient outcomes. In turn, professionals' performance is determined by a number of factors, including their own knowledge and skills. To achieve the equality requirements of health legislation, there should be a relatively even distribution of professional performance across all settings.

Health care professionals operate at a number of career levels. For doctors in the United Kingdom, the first of these is the two‐year Foundation Programme undertaken after graduation from medical school. As explained in detail below, applicants to the Foundation Programme are allocated to a geographical region, or Foundation School, based on their educational achievements—a proxy measure of their true knowledge and skills. This initial allocation of doctors is, therefore, a sensible starting point for an exploration of potential differences in performance in different geographical regions. In crude terms, if all the “best” graduates are allocated to the “best” Foundation Schools, then the differences between Foundation Schools—and potentially in the quality of patient care—are likely to be exacerbated over time, rather than reduced.

The UK Foundation Programme and application process

Upon completion of their primary medical education, typically a Bachelors in Medicine and Surgery, doctors wishing to enter clinical practice in the United Kingdom apply for Foundation Programme training posts through the United Kingdom Foundation Programme Office (UKFPO). The Foundation Programme provides structured and varied clinical practice for junior doctors prior to undertaking specialty training. It allows junior doctors to experience a variety of specialties in short rotations, typically 4 to 6 months, with an appropriate degree of training, mentorship, and supervision. , ,

Around 8000 medical students, including those from overseas medical schools wishing to undertake the Foundation Programme, are allocated across the 20 UK Foundation Schools (FS) each year. FS are based on geographical regions and there are large differences within (urban/rural and types of hospital) and between (area and population size) them. , FS oversee the administration of each junior doctor's rotations and UK Foundation Programme applications are made based on the geographical FS, before choice of specialities and job rotation are considered. A small number (<8% in 2015) of junior doctors do not go through the main Foundation Programme application process, instead entering the Foundation Programme through an alternative pathway such as the Academic Foundation Programme.

Applicants are asked to rank their preferences of FS from most to least preferred with all FS ranked. With the exception of those who do not need to apply through the UKFPO and those who have been afforded “pre‐allocation” (based on special circumstances such as caring responsibilities) students are then ranked nationally based on two performance metrics—or measures of their knowledge and skills—from their time at medical school. The first is a stand‐alone examination, the situational judgment test (SJT), which is taken in the final year of medical school and assesses attributes drawn from the job specification of a Foundation Year 1 doctor through work‐based scenarios. The second is the Educational Performance Measure (EPM), which is a composite score designed to reflect academic performance at medical school (excluding assessment performance in the final year). Points are allocated based on decile ranking against peers on various assessments within a student's own medical school (34‐43 points depending on decile; the exact “basket of assessments” used varies across medical schools), further academic achievements such as Masters‐level degrees (up to 5 points) and recognized publications (1 point per publication up to a maximum of 2). Both the EPM and SJT have a maximum score of 50 with the two scores combining to give each candidate a score out of 100.

The applicants' total scores are entered into a computer algorithm which allocates the top scoring applicant their first choice, before progressing on to the next highest scoring applicant. If an applicant's top choice is filled the algorithm considers their next preference, until an available preference can be allocated. Thus, a lower scoring applicant may have less chance of securing their first preference FS.

Whilst in their final year at medical school, applicants also sit the prescribing safety assessment (PSA). This scenario‐based examination tests a candidate's ability to prescribe medicines in a safe and competent manner. Although scores are not used in the Foundation Programme application, it is a national‐level exam which all those who wish to enter the UK Foundation Programme must attempt and thus can be used as a performance metric when assessing cohorts.

Given there are almost sufficient posts for all applicants, the UKFPO Foundation Programme application is fundamentally an allocative process, which rewards performance in EPM and SJT. Given the way the process is designed, it may result in an unequal distribution of scores between the different FS. This is because some FS may be seen as more attractive or desirable based on a number of qualities, resulting in more candidates ranking them higher. These qualities include geographic factors (current location, willingness/desire to move or links to a location), but preferences can also be strategic, based on perceived opportunities (eg, teaching, supervision, and support offered) or lack thereof.

Because FS do not have a say in which applicants are allocated to them, it is possible that the students with the most potential (the highest knowledge and skills, according to performance metrics) are concentrated in a few FS. Indeed, there is evidence that the minimum UKFPO score required to obtain a post varies between FS with, for example, a score of 84/100 required in both Central/East London and West London compared with 68/100 in Trent FS in 2018.

The aim of this research was to compare the performance of United Kingdom graduates allocated to different United Kingdom Foundation Schools according to their scores on three metrics: EPM, SJT, and PSA.

METHODS

The study was a cross‐sectional study using secondary analysis of data provided by the United Kingdom Medical Education Database (UKMED).

Data

All data were taken from the UKMED database which records the examination scores of all medical students, junior doctors and specialist trainees within the United Kingdom. The study compared data from individuals who commenced their Foundation Programme training in years 2014 to 2018 inclusive. In line with HESA standard rounding methodology, headcount variables of 0, 1, and 2 were rounded to 0. All other headcounts were rounded to the nearest multiple of 5.

Data on the FS were taken from publicly available publications released by the UKFPO. , , , , All data analysis was undertaken in STATA v15.1 and Microsoft Excel.

Analysis

Due to FS restructuring a number of FS were combined together to allow data to be compared across year groups (Table A1). This allowed us to take into account instances where FS had merged, or been separated into smaller regions and left us with 19 FS groups.

TABLE A1

Table showing School groupings used in this study

	Year
	2014	2015	2016	2017	2018
Foundation School	East Anglia	East Anglia	East Anglia	East Anglia	East Anglia
				EBH	EBH
	Leicestershire, Northamptonshire and Rutland	Leicestershire, Northamptonshire and Rutland	Leicestershire, Northamptonshire and Rutland	Leicestershire, Northamptonshire and Rutland	Leicestershire, Northamptonshire and Rutland
	North Western	North Western	North West of England	North West of England	North West of England
	Mersey	Mersey
	North Central Thames	North Central Thames	North Central Thames	North Central Thames	North Central and East London
	North East Thames	North East Thames	North East Thames	North East Thames
	North West Thames	North West Thames	North West Thames	North West Thames	North West Thames
	Northern	Northern	Northern	Northern	Northern
	Northern Ireland	Northern Ireland	Northern Ireland	Northern Ireland	Northern Ireland
	Oxford	Oxford	Oxford	Oxford	Oxford
	Peninsula	Peninsula	Peninsula	Peninsula	Peninsula
	Scotland	Scotland	Scotland	Scotland	Scotland
	Severn	Severn	Severn	Severn	Severn
	South Thames	South Thames	South Thames	South Thames	South Thames
	Trent	Trent	Trent	Trent	Trent
	Wales	Wales	Wales	Wales	Wales
	Wessex	Wessex	Wessex	Wessex	Wessex
	Staffordshire	West Midlands North	West Midlands North	West Midlands North	West Midlands North
	West Midlands, North, Central and South	West Midlands Central	West Midlands Central	West Midlands Central	West Midlands Central
	Coventry and Warwickshire	West Midlands South	West Midlands South	West Midlands South	West Midlands South
	Yorkshire and the Humber	Yorkshire and the Humber	Yorkstiire and the Humber	Yorkshire and the Humber	Yorkshire and the Humber

Applicants were then selected for eligibility based on a number of criteria: (a) United Kingdom‐trained, entering the Foundation Programme in the years 2014 to 2018, (b) did not undertake an Academic Foundation Programme, and (c) were not allocated to the Armed Forces Deanery or were otherwise removed from the UKFPO application process. This produced left us with 33 730 applicants in the study. No applicants were found following these exclusion criteria with missing data for EPM or SJT. A total of 1500 applicants were found with missing data for PSA and were excluded from this analysis. This may be due to the fact that until 2016 the PSA was not a requirement and as such not all medical schools offered the assessment to their students.

Although some graduates had multiple applications as a result of voluntary withdrawal from the process, failing final examinations or other reasons, we only used data from the application cycle in which an applicant was successful in entering the Foundation Programme. Composite elements of the EPM were totalled and this variable, along with the SJT score were confirmed as sufficiently uniformly and normally distributed within year, respectively, to allow for parametric statistical analysis. Applicants' scores at their first attempt at the PSA were used, regardless of pass or fail. Given there were multiple sits within year, with differing pass marks based on difficulty of questions, a calibration transformation was performed (using the process described by Maxwell et al ) to give a nominal pass mark of 50% Data were then confirmed as normally distributed within year. In order to allow us to compare across all year groups all three outcome measures (EPM, SJT, and PSA) were standardized within each year group with a mean of 0 and SD of 1. Following standardization, a one‐way ANOVA was performed for each outcome measure with Tukey‐Kramer a posteriori testing. The significance level for each analysis was set at 0.016, following a Bonferroni correction due to the use of three outcome measures. A “heat map” for each performance variable was constructed to show the mean standardized score within each FS, using shading that darkened with each 0.1 SD increased in the mean standardized score. Tukey‐Kramer a posteriori comparison scores were represented in table form.

Finally, we compared variation in the three outcome measures between students studying at the medical schools within each FS area (ie, before movement) and students allocated to the Foundation Programme at each FS (ie, after movement). This was done using a narrative comparison of the SD and range of the standardized means at each FS before and after movement for each outcome measure.

Power calculation

With three outcome measures we used an alpha of 0.016. EPM and SJT scores across students from United Kingdom medical schools have a mean of approximately 41/50 points and a SD of 3.5. We sought to detect a difference between the FS with the lowest and highest means of five points (equivalent to five deciles on the EPM) with all other FS with the mean score; a Cohen's f effect size of 0.143 with unequal group sizes, assuming the FS with the highest and lowest means were also the smallest FS (further details on request). Using Stata V15, we estimated we could achieve 95% power with 1800 applicants.

Ethical considerations

All data used in this study were either publicly available or provided by UKMED in an anonymized format. As such, and in line with a decision taken by Queen Mary's University of London Ethics Research Committee, no ethical approval was required.

RESULTS

The total number of applicants to the Foundation Programme in the years 2014 to 2018 was 45 075. Of these, 33 730 (75%) were included in the analysis of EPM and SJT scores and 32 230 (72%) for PSA scores. Figure 1A‐C shows a heat map with relative shading for each of the three outcome measures: (A) EPM, (B) SJT, and (C) PSA, with full results given in Table A2. FS with higher mean values of each outcome have darker shading.

FIGURE 1

Heat map of relative mean, A, educational performance measure (EPM), B, situational judgment test (SJT), and C, prescribing safety assessment (PSA) scores by Foundation School (FS). The three FS in London are shown in the top left corner. The key to the mean score from the shading used is the same for all three variables, as shown in the legend (in SD units). SD, standard deviation

TABLE A2

Mean standardized scores by Foundation School

Foundation School	Code on map	Number of students		Educational performance measure (EPM)		Situational judgment test (SJT)		Prescribing safety assessment (PSA)
		EPM/SJT	PSA	Standardized mean	95% CI	Standardized mean	95% CI	Standardized mean	95% CI
East Anglia	1	1845	1700	−0.232	−0.272 to −0.192	−0.284	−0.327 to −0.241	−0.151	−0.199 to −0.103
Leicestershire, Northamptonshire and Rutland (LNR)	3	725	690	−0.417	−0.482 to −0.352	−0.353	−0.423 to −0.283	−0.342	−0.418 to −0.267
North West of England	6	3910	3805	−0.073	−0.103 to −0.044	0.016	−0.013 to 0.045	−0.034	−0.064 to −0.003
North Central and East London	4	2480	2300	0.638	0.607 to 0.669	0.427	0.395 to 0.460	0.344	0.306 to 0.382
North West Thames	5	1195	1155	1.069	1.029 to 1.110	0.684	0.639 to 0.730	0.485	0.434 to 0.536
Northern	7	1805	1745	−0.412	−0.458 to −0.366	−0.268	−0.321 to −0.216	−0.204	−0.253 to −0.154
Northern Ireland	8	1180	1145	−0.204	−0.260 to −0.148	0.055	0.001 to 0.108	0.135	0.078 to 0.192
Oxford	9	1030	970	0.491	0.444 to 0.539	0.324	0.275 to 0.372	0.307	0.250 to 0.363
Peninsula	10	915	890	−0.278	−0.338 to −0.218	−0.041	−0.102 to 0.021	−0.051	−0.113 to 0.012
Scotland	11	3765	3535	−0.123	−0.155 to −0.091	−0.026	−0.059 to 0.006	−0.155	−0.188 to −0.122
Severn	12	1280	1255	0.706	0.663 to 0.750	0.497	0.453 to 0.541	0.387	0.338 to 0.437
South Thames	13	3820	3585	0.378	0.352 to 0.403	0.235	0.209 to 0.261	0.160	0.129 to 0.190
Trent	14	1385	1330	−0.444	−0.494 to −0.395	−0.511	−0.569 to −0.453	−0.302	−0.354 to −0.249
Wales	15	1530	1490	−0.348	−0.398 to −0.297	−0.291	−0.343 to −0.240	−0.143	−0.192 to −0.093
Wessex	16	1400	1345	−0.083	−0.128 to −0.039	0.026	−0.020 to 0.072	−0.057	−0.109 to −0.005
W Mids North	18	1005	960	−0.854	−0.909 to −0.798	−0.903	−0.974 to −0.832	−0.451	−0.519 to −0.382
W Mids Central	17	1085	1070	−0.040	−0.093 to 0.013	0.069	0.016 to 0.123	0.158	0.100 to 0.217
W Mids South	19	665	640	−0.314	−0.384 to −0.244	−0.320	−0.395 to −0.244	−0.128	−0.207 to −0.049
Yorkshire and the Humber	2	2710	2630	−0.230	−0.268 to −0.193	−0.115	−0.153 to −0.077	−0.097	−0.136 to −0.059

The one‐way ANOVA comparing EPM across FS gave an F value of 401 (P < .001). This shows a statistically significant difference between FS. The variation in mean scores between FS is equivalent to approximately six EPM points between the means at the highest (NW Thames) and lowest (W Midlands North) FS. Given that the minimum possible score is 34/50 and the ensuing overall range of 16 points this represents a relative difference of 37%. The one‐way ANOVA comparing SJT across FS gave an F value of 213 (P < .001). Again this shows a statistically significant difference between FS. The difference between FS with the highest (NW Thames) and lowest (W Midlands North) means is equivalent to 6/50 SJT points (12%). The one‐way ANOVA comparing PSA scores across FS gave an F value of 95.4 (P < .001). This also shows a statistically significant difference between FS. The difference in calibrated PSA scores between the FS with the highest (NW Thames) and lowest (W Midlands North) means is equivalent to 11 points (11%).

Table 1 summarizes the results of the 171 pairwise comparisons for each of the three performance metrics. Over three‐quarters of the pairwise comparisons were statistically significant, although a lower proportion were of at least a “medium” effect size: 47% for the EPM, 36% for the SJT and 20% for the PSA. Full results are shown in Tables A3 and A4.

TABLE 1

Pairwise comparisons between Foundation Schools

		EPM	SJT	PSA
	N (%) statistically significant differences (Tukey's test)	137 (80%)	140 (82%)	123 (72%)
N (%) of comparisons with an absolute difference in standardized means of:	<0.2 (No effect)	55 (32%)	44 (26%)	67 (39%)
	0.2 to 0.499 (Small effect)	37 (22%)	65 (38%)	71 (42%)
	0.5 to 0.799 (Medium effect)	35 (21%)	40 (23%)	30 (18%)
	≥ 0.8 (Large effect)	44 (26%)	33 (13%)	3 (2%)

Abbreviations: EPM, educational performance measure; PSA, prescribing safety assessment; SJT, situational judgment test.

TABLE A3

Composite table showing Tukey test outcomes between FS for EPM, PSA, and SJT

	East Anglia
LNR	−4.35	0.002	LNR												Foundation School	t value	P > |t|
	−4.66	<0.001														PSA t	PSA P > |t|
	−1.66	0.979														EPM t	EPM P > |t|
North West of England	4.13	0.005	7.66	0	North West of England											SJT t	SJT P > |t|
	6.19	<0.001	9.38	0
	11.21	<0.001	9.64	0
North Central and East London	15.88	<0.001	16.24	0	14.66	0	North Central and East London
	31.16	<0.001	27.56	0	30.52	0
	24.4	<0.001	19.52	0	16.9	0
North West Thames	17.13	<0.001	17.66	0	15.85	0	4.02	0.008	North West Thames
	38.62	<0.001	34.83	0	38.1	0	13.51	0
	27.53	<0.001	23.28	0	21.35	0	7.72	0
Northern	−1.58	0.988	3.17	0.142	−6.04	0	−17.7	0	−18.64	0	Northern
	−6	<0.001	0.12	1	−13.13	0	−37.4	0	−43.79	0
	0.51	1	2.04	0.87	−10.54	0	−23.71	0	−26.97	0
Northern Ireland	7.69	<0.001	10.18	0	5.14	0	−5.92	0	−8.61	0	9.14	0	Northern Ireland
	0.82	1	4.98	0	−4.34	0.002	−26.23	0	−34.2	0	6.13	0
	9.58	<0.001	9.12	0	1.22	0.999	−11.12	0	−16.21	0	9.1	0
Oxford	11.68	<0.001	13.39	0	9.71	0	−0.99	1	−4.2	0.004	13.08	0	4.04	0.007	Oxford
	20.48	<0.001	20.66	0	17.75	0	4.36	0.002	14.99	0	25.49	0	17.96	0
	16.48	<0.001	14.74	0	9.27	0	−2.94	0.245	−8.96	0	15.99	0	6.66	0
Peninsula	2.49	0.566	5.9	0	−0.47	1	−10.24	0	−12.31	0	3.81	0.018	−4.26	0.003	−7.9	0	Peninsula
	−1.27	0.999	3.08	0.178	−6.16	0	−26.12	0	−33.83	0	3.64	0.033	−1.86	0.938	−18.67	0
	6.36	<0.001	6.64	0	−1.63	0.983	−12.77	0	−17.43	0	5.92	0	−2.28	0.725	−8.46	0
Scotland	−0.13	1	4.63	0.001	−5.32	0	−19.1	0	−19.38	0	1.71	0.972	−8.75	0	−13.07	0	−2.85	0.303	Scotland
	4.22	0.004	8	0	−2.4	0.632	−32.42	0	−39.59	0	11.13	0	2.67	0.425	−19.25	0	4.65	0.001
	9.56	<0.001	8.51	0	−1.96	0.904	−18.5	0	−22.6	0	8.91	0	−2.56	0.512	−10.5	0	0.41	1
Severn	14.85	<0.001	15.81	0	13.27	0	1.28	0.999	−2.45	0.596	16.39	0	6.34	0	1.94	0.914	10.25	0	16.93	0	Severn
	28.41	<0.001	26.65	0	26.67	0	2.19	0.786	−9.95	0	33.73	0	24.85	0	5.66	0	25.07	0	28.24	0
	22.65	<0.001	19.31	0	15.76	0	2.14	0.817	−4.93	0	22.1	0	11.57	0	4.37	0.002	13.11	0	17.07	0
South Thames	10.83	<0.001	12.41	0	8.52	0	−7.07	0	−9.87	0	12.78	0	0.74	1	−4.17	0.004	5.76	0	13.62	0	−7.13	0	South Thames
	23.67	<0.001	21.64	0	21.83	0	−11.12	0	−23.01	0	30.47	0	19.24	0	−3.57	0.043	19.65	0	24.02	0	−11.21	0
	19.3	<0.001	15.33	0	10.14	0	−7.87	0	−14.32	0	18.58	0	5.71	0	−2.67	0.426	7.9	0	12	0	−8.56	0
Trent	−4.22	0.004	0.89	1	−8.63	0	−19.22	0	−20.07	0	−2.76	0.362	−11.11	0	−14.78	0	−5.94	0	−4.68	0	−17.96	0	−14.74	0	Trent
	−6.59	<0.001	−0.66	1	−13.09	0	−35.56	0	−42.27	0	−1	1	−6.69	0	−25.06	0	−4.3	0.003	−11.27	0	−32.71	0	−28.88	0
	−6.73	0	−3.63	0.034	−17.78	0	−29.51	0	−31.96	0	−7.17	0	−15.06	0	−21.4	0	−11.65	0	−16.27	0	−27.43	0	−25.08	0
Wales	0.25	1	4.45	0.001	−3.66	0.031	−15.01	0	−16.43	0	1.78	0.96	−7.25	0	−11.18	0	−2.22	0.763	0.41	1	−14.19	0	−10.06	0	4.32	0.002	Wales
	−3.69	0.028	1.7	0.974	−10.03	0	−33.4	0	−40.46	0	2.05	0.865	−4.09	0.006	−22.93	0	−1.83	0.948	−8.16	0	−30.66	0	−26.41	0	2.88	0.284
	0.23	1	1.44	0.996	−10.76	0	−23.32	0	−26.68	0	−0.71	1	−9.42	0	−16.1	0	−6.34	0	−9.23	0	−21.96	0	−18.35	0	6.24	0
Wessex	2.65	0.445	6.26	0	−0.76	1	−11.98	0	−13.87	0	4.15	0.005	−4.91	0	−8.87	0	−0.15	1	3.13	0.155	−11.62	0	−6.96	0	6.49	0	2.33	0.686	Wessex
	4.61	0.001	8.04	0	−0.36	1	−23.76	0	−32.25	0	10.17	0	3.36	0.081	−15.42	0	5.06	0	1.4	0.997	−22.49	0	−16.25	0	10.5	0	7.87	0
	9.23	<0.001	8.75	0	0.34	1	−12.65	0	−17.64	0	8.72	0	−0.75	1	−7.64	0	1.66	0.98	1.77	0.961	−12.84	0	−7.04	0	14.95	0	9.06	0
W Mids North	−7.6	0	−2.23	0.762	−11.83	0	−21.19	0	−21.97	0	−6.3	0	−13.72	0	−17.06	0	−8.8	0	−8.33	0	−20.03	0	−17.21	0	−3.6	0.037	−7.63	0	−9.55	0	W Mids North
	−17.46	0	−9.87	0	−24.29	0	−43.92	0	−49.49	0	−12.35	0	−16.67	0	−33.4	0	−13.87	0	−22.66	0	−40.76	0	−38.23	0	−10.87	0	−13.72	0	−20.51	0
	−16.65	0	−11.91	0	−27.4	0	−37.51	0	−39.13	0	−17.01	0	−23.53	0	−29.18	0	−19.92	0	−26.03	0	−35.03	0	−33.84	0	−9.98	0	−15.88	0	−23.7	0
W Mids Central	8.14	<0.001	10.53	0	5.7	0	−5.14	0	−7.9	0	9.57	0	0.56	1	−3.44	0.065	4.73	0	9.22	0	−5.65	0	−0.04	1	11.49	0	7.71	0	5.4	0	14.05	0	W Mids Central
	5.52	<0.001	8.67	0	1.06	1	−20.53	0	−29.17	0	10.68	0	4.3	0.003	−13.46	0	5.85	0	2.65	0.438	−19.94	0	−13.38	0	11	0	8.54	0	1.18	1	20.47	0
	9.75	<0.001	9.3	0	1.64	0.982	−10.38	0	−15.49	0	9.28	0	0.37	1	−6.17	0	2.59	0.489	2.93	0.252	−10.94	0	−5.08	0	15.11	0	9.6	0	1.13	1	23.43	0
W Mids South	0.52	1	4.02	0.008	−2.26	0.738	−10.83	0	−12.77	0	1.69	0.976	−5.47	0	−8.76	0	−1.53	0.992	0.65	1	−10.89	0	−6.88	0	3.71	0.026	0.32	1	−1.51	0.993	6.49	0	−5.88	0	W Mids South
	−2	0.886	2.11	0.83	−6.32	0	−24	0	−31.5	0	2.38	0.649	−2.5	0.555	−17.83	0	−0.77	1	−5	0	−23.51	0	−18.12	0	3.04	0.194	0.8	1	−5.39	0	11.88	0	−6.13	0
	−0.83	1	0.66	1	−8.44	0	−18.03	0	−21.89	0	−1.2	1	−8.14	0	−13.64	0	−5.78	0	−7.35	0	−18.01	0	−13.91	0	4.27	0.003	−0.64	1	−7.74	0	12.31	0	−8.33	0
Yorkshire and the Humber	1.78	0.959	5.89	0	−2.57	0.501	−15.85	0	−16.93	0	3.54	0.046	−6.74	0	−11.04	0	−1.23	0.999	2.3	0.711	−14.49	0	−10.12	0	6.23	0	1.44	0.996	−1.23	0.999	9.61	0	−7.24	0	0.71	1
	0.05	1	4.92	0	−6.93	0	−34.42	0	−41.26	0	6.59	0	−0.84	1	−21.72	0	1.38	0.998	−4.7	0	−30.43	0	−26.67	0	7.14	0	4.04	0.008	−4.92	0	18.58	0	−5.84	0	2.13	0.823
	5.89	<0.001	6	0	−5.55	0	−20.59	0	−24.31	0	5.31	0	−5.14	0	−12.65	0	−2.06	0.857	−3.73	0.025	−19.05	0	−14.7	0	12.63	0	5.81	0	−4.54	0.001	22.49	0	−5.43	0	4.98	0

Note: Results are shown in bold where the comparison is statistically significant and the FS in the vertical column has the higher average. Results are shown in italics where the comparison is statistically significant and the FS listed horizontally has the higher average.

TABLE A4

Composite table showing pairwise comparisons between FS

	East Anglia
LNR	0.19	LNR
	0.19
	0.07
North West of England	−0.12	−0.31	North West of England								Foundation School 2
	−0.16	−0.34								Foundation School 1	PSA
	−0.30	−0.37									EPM
North Central and East London	−0.49	−0.69	−0.38	North Central and East London							SJT
	−0.87	−1.05	−0.71
	−0.71	−0.78	−0.41
North West Thames	−0.64	−0.83	−0.52	−0.14	North West Thames
	−1.30	−1.49	−1.14	−0.43
	−0.97	−1.04	−0.67	−0.26
Northern	0.05	−0.14	0.17	0.55	0.69	Northern
	0.18	0.00	0.34	1.05	1.48
	−0.02	−0.08	0.28	0.70	0.95
Northern Ireland	−0.29	−0.48	−0.17	0.21	0.35	−0.34	Northern Ireland
	−0.03	−0.21	0.13	0.84	1.27	−0.21
	−0.34	−0.41	−0.04	0.37	0.63	−0.32
Oxford	−0.46	−0.65	−0.34	0.04	0.18	−0.51	−0.17	Oxford
	−0.72	−0.91	−0.56	0.15	0.58	−0.90	−0.70
	−0.61	−0.68	−0.31	0.10	0.36	−0.59	−0.27
Peninsula	−0.10	−0.29	0.02	0.39	0.54	−0.15	0.19	0.36	Peninsula
	0.05	−0.14	0.21	0.92	1.35	−0.13	0.07	0.77
	−0.24	−0.31	0.06	0.47	0.73	−0.23	0.10	0.36
Scotland	0.00	−0.19	0.12	0.50	0.64	−0.05	0.29	0.46	0.10	Scotland
	−0.11	−0.29	0.05	0.76	1.19	−0.29	−0.08	0.61	−0.16
	−0.26	−0.33	0.04	0.45	0.71	−0.24	0.08	0.35	−0.01
Severn	−0.54	−0.73	−0.42	−0.04	0.10	−0.59	−0.25	−0.08	−0.44	−0.54	Severn
	−0.94	−1.12	−0.78	−0.07	0.36	−1.12	−0.91	−0.21	−0.98	−0.83
	−0.78	−0.85	−0.48	−0.07	0.19	−0.76	−0.44	−0.17	−0.54	−0.52
South Thames	−0.31	−0.50	−0.19	0.18	0.33	−0.36	−0.02	0.15	−0.21	−0.31	0.23	South Thames
	−0.61	−0.79	−0.45	0.26	0.69	−0.79	−0.58	0.11	−0.66	−0.50	0.33
	−0.52	−0.59	−0.22	0.19	0.45	−0.50	−0.18	0.09	−0.28	−0.26	0.26
Trent	0.15	−0.04	0.27	0.65	0.79	0.10	0.44	0.61	0.25	0.15	0.69	0.46	Trent
	0.21	0.03	0.37	1.08	1.51	0.03	0.24	0.94	0.17	0.32	1.15	0.82
	0.23	0.16	0.53	0.94	1.20	0.24	0.57	0.83	0.47	0.48	1.01	0.75
Wales	−0.01	−0.20	0.11	0.49	0.63	−0.06	0.28	0.45	0.09	−0.01	0.53	0.30	−0.16	Wales
	0.12	−0.07	0.27	0.99	1.42	−0.06	0.14	0.84	0.07	0.22	1.05	0.73	−0.10
	0.01	−0.06	0.31	0.72	0.98	0.02	0.35	0.62	0.25	0.27	0.79	0.53	−0.22
Wessex	−0.09	−0.29	0.02	0.40	0.54	−0.15	0.19	0.36	0.01	−0.10	0.44	0.22	−0.24	−0.09	Wessex
	−0.15	−0.33	0.01	0.72	1.15	−0.33	−0.12	0.57	−0.19	−0.04	0.79	0.46	−0.36	−0.26
	−0.31	−0.38	−0.01	0.40	0.66	−0.29	0.03	0.30	−0.07	−0.05	0.47	0.21	−0.54	−0.32
W Mids North	0.30	0.11	0.42	0.79	0.94	0.25	0.59	0.76	0.40	0.30	0.84	0.61	0.15	0.31	0.39	W Mids North
	0.62	0.44	0.78	1.49	1.92	0.44	0.65	1.34	0.58	0.73	1.56	1.23	0.41	0.51	0.77
	0.62	0.55	0.92	1.33	1.59	0.63	0.96	1.23	0.86	0.88	1.40	1.14	0.39	0.61	0.93
W Mids Central	−0.31	−0.50	−0.19	0.19	0.33	−0.36	−0.02	0.15	−0.21	−0.31	0.23	0.00	−0.46	−0.30	−0.22	−0.61	W Mids Central
	−0.19	−0.38	−0.03	0.68	1.11	−0.37	−0.16	0.53	−0.24	−0.08	0.75	0.42	−0.40	−0.31	−0.04	−0.81
	−0.35	−0.42	−0.05	0.36	0.62	−0.34	−0.01	0.25	−0.11	−0.10	0.43	0.17	−0.58	−0.36	−0.04	−0.97
W Mids South	−0.02	−0.21	0.09	0.47	0.61	−0.08	0.26	0.43	0.08	−0.03	0.52	0.29	−0.17	−0.01	0.07	−0.32	0.29	W Mids South
	0.08	−0.10	0.24	0.95	1.38	−0.10	0.11	0.81	0.04	0.19	1.02	0.69	−0.13	−0.03	0.23	−0.54	0.27
	0.04	−0.03	0.34	0.75	1.00	0.05	0.37	0.64	0.28	0.29	0.82	0.55	−0.19	0.03	0.35	−0.58	0.39
Yorkshire and the Humber	−0.05	−0.25	0.06	0.44	0.58	−0.11	0.23	0.40	0.05	−0.06	0.48	0.26	−0.20	−0.05	0.04	−0.35	0.26	−0.03
	0.00	−0.19	0.16	0.87	1.30	−0.18	0.03	0.72	−0.05	0.11	0.94	0.61	−0.21	−0.12	0.15	−0.62	0.19	−0.08
	−0.17	−0.24	0.13	0.54	0.80	−0.15	0.17	0.44	0.07	0.09	0.61	0.35	−0.40	−0.18	0.14	−0.79	0.18	−0.20

Note: Results are highlighted based on effect size with negative values still representing a greater effect. Row values are compared to column values, for example, LNR > East Anglia in all three measures, Northern < Scotland in all three measures.

Table 2 shows the comparison of the variability in each outcome measure between students studying at the medical schools within each FS area and students allocated to the Foundation Programme at each FS. A posteriori ANOVAs identified statistically significant differences between the mean scores of students at medical schools across FS areas for all three outcomes (all P < .001). Nevertheless, the results in Table 2 suggest that the Foundation Programme application process and subsequent student movement led to an increased variability in mean scores based on EPM and SJT performance, but a reduced variability based on PSA performance.

TABLE 2

The effect of student movement on variability between Foundation Schools

	EPM		SJT		PSA
	Medical schools in FS area	FS	Medical schools in FS area	FS	Medical schools in FS area	FS
SD of means	0.22	0.48	0.14	0.48	0.29	0.26
Lowest mean	−0.41	−0.86	−0.14	−0.85	−0.54	−0.45
Highest mean	0.39	1.07	0.48	1.07	0.68	0.49
Range in means	0.79	1.92	0.62	1.92	1.22	0.94

Abbreviations: EPM, educational performance measure; PSA, prescribing safety assessment; SJT, situational judgment test.

DISCUSSION

Our results show that there is significant variation between FS in all three performance metrics we have considered: the difference in mean scores between the highest and lowest scoring FS on all three metrics would be considered a “large” effect size (>0.8) when using Cohen's rules of thumb. The variation is greatest for the EPM and least for the PSA. When considering the differences between the highest and lowest scoring FS on the EPM, the difference is equivalent to the average Foundation doctor having been in the first vs the sixth decile at medical school. The heat maps show that the three performance metrics are consistent across the FS (with, for example, Severn and North Central and East London scoring highly in all three variables and West Midlands North, Leicestershire, Northamptonshire and Rutland, and Trent all scoring poorly). Our narrative analysis of the effect of student movement on mean scores in each FS area suggests that movement (resulting from the Foundation Programme application process) exacerbates regional differences in performance for two of the three outcome measures (EPM and SJT, but not PSA).

Given the importance of doctors' well‐being, ignoring students' location preferences altogether to create an equal distribution based on metrics of knowledge and skills would be futile. In 2018, 77% of students were allocated to their first choice of FS and 95% to one of their top five, suggesting the current system appears to be working fairly well from a student location choice perspective. Understanding students' preferences and how these are formed is, therefore, important. Preferences may be driven by location, perceived training quality and social relationships, as well as demographic variables such as gender and ethnicity. FS areas are large, and include a range of types of hospital and generally, both urban and rural areas. Students will rotate round several placements during their two‐year Foundation Programme, so it is difficult to determine the influence of hospital and location type using the data analyzed in this study, although this is an important avenue for further research.

More detailed study of how less competitive FS could be made more attractive would also be useful. It is worth noting that for the 2019/2020 and 2020/2021 intakes Geographical Foundation Priority Programmes (FPP) have been offered to help recruit junior doctors to areas which struggle to recruit and retain junior doctors and specialist trainees. Furthermore a number of FS are offering specialist Foundation Priority Programmes, tailored toward a particular career choice. Both incentives may go some way to addressing the differences between FS that we have identified.

Despite the large number of applicants in this study, it is not without its limitations. Perhaps the first, and most pertinent, limitation to address is that this study does not directly equate performance in the measured metrics of knowledge and skills to patient care outcomes. Second, it is worth recognizing that due to the changes in FS footprint over the years of the study we have had to combine some FS for our analysis. Whilst we recognize that this may have affected the between FS interpretations to a small degree, we do not believe that this affects the overall finding that significant variation exists between the scores of junior doctors allocated to different FS. Third, we have not separated EPM deciles from scores for other educational achievements and therefore cannot say which component(s) contribute to the differences reported here. Finally, it has not been possible to include the small number (590 in 2018) of individuals who completed their initial training outside of the United Kingdom and have chosen to commence working within the Foundation Programme. We consider it very unlikely that these individuals would have had an impact on our overall findings.

The potential implications of our findings would only have consequences for patient care if the three metrics are good predictors of performance in the Foundation Programme and of the quality of care provided more generally. There is evidence to suggest that the SJT and EPM are accurate predictors of performance, as reported by senior doctors, for a doctor in the Foundation Programme We have found no research linking performance in SJT, EPM, or PSA to patient outcomes or direct measures of quality of care, although Archer et al's systematic review of the impact of licensing examinations does find a positive correlation between performance in these exams and “some patient outcomes and rates of complaints.” A companion study to this one reports that EPM totals/deciles are predictive of the hazard of having a sanction imposed by the GMC, but SJT scores are not.

In addition to the FPP now being introduced, we suggest that there are three main ways that the statistically significant difference between FS could be addressed. All of these ways are controversial, and none should be implemented without further research linking medical school and postgraduate clinical performance. The first, and perhaps most simple is the provision of financial (or other) rewards to high performing applicants to FS that have a lower mean. Evidence suggests that this is a successful strategy for encouraging applicants to apply to these regions. , The second would be to prioritize resources to FS with lower means to ensure that the junior doctors in these schools were the most supported and supervised, helping to raise the lower end of the performance distribution. The third and most drastic would be to fundamentally overhaul the Foundation Programme application system to ensure a fair geographical distribution of caliber of candidates. Candidates would continue to rank their preferences as before but would then be randomly nationally ranked. Foundation Programme training posts would then be awarded based on this random ranking, taking into account the applicants' preferences and remaining available posts.

CONCLUSION

The aim of this research was to determine whether the UKFPO application process results in differences in the mean EPM, SJT, and PSA scores of students allocated to different FS. We can conclude that there is a difference in the knowledge and skills of junior doctors entering the Foundation Programme based on geographical location as measured by all three metrics. Together with concurrent research on the predictive value of EPM scores/deciles on fitness to practise sanctions imposed by the GMC, our findings may suggest a variation in the quality of patient care provided which would constitute, from the perspective of the patient, an inherent unfairness in the way allocations are made. If applicants “vote with their feet” toward FS that are perceived to offer better quality training and supervision because they offer higher quality patient care, then these differences in quality of care—and health inequalities—could be exacerbated in the long term. Our research should provide the basis for further, more detailed analysis of the implication of performance metrics used in selection and allocation on patient care.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

AUTHOR CONTRIBUTIONS

Conceptualization: Celia Brown

Data curation: Christopher Beck

Formal analysis: Christopher Beck, Celia Brown

Investigation: Christopher Beck, Celia Brown

Methodology: Celia Brown

Project administration: Christopher Beck

Software: Christopher Beck, Celia Brown

Supervision: Celia Brown

Visualization: Christopher Beck

Writing ‐ original draft preparation: Christopher Beck

Writing ‐ review and editing: Celia Brown

All authors have read and approved the final version of the manuscript.

Celia Brown had full access to all of the data in this study and takes complete responsibility for the integrity of the data and the accuracy of the data analysis.

TRANSPARENCY STATEMENT

Celia Brown affirms that this manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.