Widening access to UK medical education for under-represented socioeconomic groups: modelling the impact of the UKCAT in the 2009 cohort.

OBJECTIVE: To determine whether the use of the UK clinical aptitude test (UKCAT) in the medical schools admissions process reduces the relative disadvantage encountered by certain sociodemographic groups.
DESIGN: Prospective cohort study.
SETTING: Applicants to 22 UK medical schools in 2009 that were members of the consortium of institutions utilising the UKCAT as a component of their admissions process. PARTICIPANTS: 8459 applicants (24,844 applications) to UKCAT consortium member medical schools where data were available on advanced qualifications and socioeconomic background. MAIN OUTCOME MEASURES: The probability of an application resulting in an offer of a place on a medicine course according to seven educational and sociodemographic variables depending on how the UKCAT was used by the medical school (in borderline cases, as a factor in admissions, or as a threshold).
RESULTS: On univariate analysis all educational and sociodemographic variables were significantly associated with the relative odds of an application being successful. The multilevel multiple logistic regression models, however, varied between medical schools according to the way that the UKCAT was used. For example, a candidate from a non-professional background was much less likely to receive a conditional offer of a place compared with an applicant from a higher social class when applying to an institution using the test only in borderline cases (odds ratio 0.51, 95% confidence interval 0.45 to 0.60). No such effect was observed for such candidates applying to medical schools using the threshold approach (1.27, 0.84 to 1.91). These differences were generally reflected in the interactions observed when the analysis was repeated, pooling the data. Notably, candidates from several under-represented groups applying to medical schools that used a threshold approach to the UKCAT were less disadvantaged than those applying to the other institutions in the consortium. These effects were partially reflected in significant differences in the absolute proportion of such candidates finally taking up places in the different types of medical schools; stronger use of the test score (as a factor or threshold) was associated with a significantly increased odds of entrants being male (1.74, 1.25 to 2.41) and from a low socioeconomic background (3.57, 1.03 to 12.39). There was a non-significant trend towards entrants being from a state (non-grammar) school (1.60, 0.97 to 2.62) where a stronger use of the test was employed. Use of the test only in borderline cases was associated with increased odds of entrants having relatively low academic attainment (5.19, 2.02 to 13.33) and English as a second language (2.15, 1.03 to 4.48).
CONCLUSIONS: The use of the UKCAT may lead to more equitable provision of offers to those applying to medical school from under-represented sociodemographic groups. This may translate into higher numbers of some, but not all, relatively disadvantaged students entering the UK medical profession.

Introduction

Since the Further and Higher Education Act was introduced in 1992 efforts have been made to improve social mobility through access to higher education. Nevertheless, access to the professions remains largely restricted to those from relatively advantaged backgrounds. In particular the situation in medicine remains largely unchanged since the 1970s, with only around 5% of entrants having parents from a non-professional background.1 2 This is in part a problem of application rates.3 None the less, the success rate is lower for equivalent candidates from different class backgrounds.4 This disparity may be at least partly due to the use of A levels (or equivalent advanced qualifications) as a primary selection tool for ability within the United Kingdom, as A levels are known to advantage applicants from privileged backgrounds.5 A high degree of subjectivity has been shown in the reasons given by selectors for short listing candidates before interview.6 Personal statements may also be relied on, which tend to favour pupils from independent schools,7 and have been viewed as a poor way of selecting candidates for medical school. In addition, face-to-face interviews may add little information to the admissions process, having been shown to be only weakly predictive of subsequent academic performance.8

Selection bias is problematic for several reasons. Firstly, if candidates of equal ability are disadvantaged by their socioeconomic or school background then this is unjust and unacceptable within society as a whole. Secondly, at least some evidence shows that doctors from particular social backgrounds can perform a particular service in dealing with the health needs in their original communities.9 Thirdly, diversity in student populations can promote positive awareness of the value of diversity in society.10

Attempts have been made to deal with this situation through several initiatives. These include the expansion of access courses for those without the requisite examination qualifications in the sciences and graduate entry courses. Access courses have resulted in an increased proportion of medical students from backgrounds that differ from those of students traditionally entering medicine, but the overall impact has been small. Moreover, although graduate entrants are from a more representative range of socioeconomic backgrounds11 these programmes have similarly not led to the significant changes in the socioeconomic profile of UK medical students.12 It is therefore clear that new, wider approaches are required and must prove their effectiveness. This desire to widen participation in medicine was partly the driver behind the development of the UK clinical aptitude test (UKCAT), introduced in 2006. The test, developed in detail and delivered by a computer based testing and assessment business,13 was intended to evaluate abilities (on an experimental basis) that were considered pertinent to the success of a medical career.14 The UKCAT scores are only modestly correlated with attainment at A level examinations and are significantly associated with several sociodemographic variables, although the scores may be less sensitive to the school type attended than to academic achievement.15 16 Nevertheless, the constructs measured by the test may be sufficiently different from those estimated by traditional educational attainment to tackle some of the relative disadvantages encountered by certain sociodemographic groups, such as those from state schools rather than independent schools.

The use of the UKCAT as a component of the medical selection process has attracted considerable criticism, including that of the BMA students committee. Opponents of the test highlight that A level achievement is predictive of medical performance at undergraduate and postgraduate level, whereas the evidence for the predictive validity of the UKCAT is currently lacking.17 Moreover, it has been suggested that the fee of £65 (€78; $102) to £100 for taking the UKCAT may actually discourage those from less economically advantage backgrounds from applying to medical school.18 Consequently it is important to carry out research to establish whether including the UKCAT as a component of the selection process adds value beyond that provided by traditional measures of academic attainment.

In the 2009 round of applications for admission to medical school in 2010, 25 of the 31 UK based medical schools utilised the UKCAT results as part of their admissions process. We hypothesised that candidates from certain sociodemographic groups applying to medical schools that utilised the UKCAT scores relatively heavily as part of their admissions processes would experience less disadvantage compared with those applying to institutions with a relatively weak use of the test. More specifically, we proposed that those using performance on the UKCAT as a “threshold” score would show the greatest impact on reducing the relative disadvantage for certain under-represented groups of candidates. We also anticipated that those institutions using the test as a factor in the admissions process would have a more noticeable effect compared with those using the test only in borderline cases.

By using data from the 2009 application cohort we planned to develop and populate models estimating the probability of an application being successful, given several available sociodemographic and educational variables and the manner in which UKCAT was used by the medical schools. Moreover, we intended to utilise information relating to final placement to evaluate to what extent, if any, trends in the provision of offers was reflected in the sociodemographic characteristics of those eventually taking up places at the three categories of medical school. Consequently we aimed to estimate the potential final impact on the demographic make-up of UK doctors of any differences between groups in the medical schools’ behaviours relating to making offers.

Methods

Data were available for candidates who applied during 2009 to one or more of the 25 UK based medical schools that were members of the consortium of institutions utilising the UKCAT as a component of their admissions process. We excluded applications specifically relating to graduate entry or “widening participation” courses as they were considered a potential source of confounding. Thus we excluded data from three of the consortium medical schools as these institutions exclusively offered these medical courses, leaving 22 participating institutions. At the outset we also excluded a small number of candidates with incomplete or missing UKCAT scores (n=227). In some cases this missingness may have been due to the unavailability of UKCAT centres in certain countries. At registration for the UKCAT, students were informed that the information would be used for educational research and evaluation of the UKCAT and that the results would be published in a form such that individual students could not be identified. Thus the data were routinely contemporaneously collected and anonymised. We grouped the participating universities into three categories (box) according to their predominant use of the test, as reported to the UKCAT consortium board during the annual telephone survey of medical schools.19

UKCAT score used in borderline cases, as a tie-breaker, or (more rarely) as a “rescue” mechanism to offer interviews to candidates whose applications would otherwise rate poorly—that is, weak use of the test or few candidates affected (n=6)

UKCAT used as a factor in deciding to interview or offer, or both (moderate use of the test). In practice this meant 4-33% of the decision to offer an interview or place was based on UKCAT scores (n=9)

A threshold score used to decide whether to offer an interview (relatively strong use of test) (n=7). Thresholds ranged from a total score of 1900 to 2730. Only two schools set the threshold scores before the start of the admissions process

We designed this modelling process to test whether these classes of medical schools could be ranked meaningfully in relation to sociodemographic factors. Occasionally an institution used the test as both a factor and a threshold in selection. In these cases we chose the strongest mode of use to categorise the medical school.

Data preparation

All data used in the analyses were abstracted in anonymous form from the combined Universities and Colleges Admissions Service (UCAS) and UKCAT database. Figure 1 depicts the data included in the final modelling process. The structure of offer data is complex in that information relates to both individual candidates and specific applications to each institution. The dependent variable was the outcome (offer or rejection) of an “application event” involving one candidate and one specific medical school. To each of these application events we attributed a unique identifier. We included both conditional (dependent on attainment of A level or equivalent examinations) and unconditional offers (academic requirements for entry fulfilled) as a positive outcome and analysed these separately, because unconditional offers also depended on a candidate with a conditional offer obtaining the required grades at A level or equivalent, and therefore these outcomes were likely to differ in their predictors. We summed and converted the UKCAT subtest scores to standardised z scores to create an overall metric of UKCAT test ability, in comparison with a student’s peers. Table 1 summarises the independent variables utilised in the modelling process. Male sex was also included as a disadvantaged category for two reasons; there is clear evidence that females are over-represented in medical school intakes compared with the UK population,20 21 and univariate analysis showed that male sex was associated with a reduced odds of an offer. We used both standardised UKCAT and A level (or equivalent) tariff scores in both a continuous and a dichotomous form, although not in the same analyses owing to problems of multicollinearity. All data management and analyses were carried out using STATA MP version 12.22

Fig 1 Flow of data in analyses. Observations with missing data were included in preliminary univariate analysis, although not in the final model’s development

Table 1

 Sociodemographic and educational characteristics of applicants by groups of medical school applied to according to use of the UK clinical aptitude test (UKCAT). Values are mean (standard deviation) percentages of applicants from widening participation group unless stated otherwise

Widening participation group (% missing)	UKCAT in borderline cases (6 medical schools, 10 086 applicants)	UKCAT as factor (9 medical schools, 11 584 applicants)	UKCAT as threshold (7 medical schools, 10 208 applicants)	Odds ratio (95% CI): factor v borderline use	P value	Odds ratio (95% CI): threshold v factor use	P value	Odds ratio (95% CI): threshold v borderline use	P value
Males (0%)	44.9 (3.6)	44.6 (2.5)	46.9 (2.5)	1.00 (0.95 to 1.04)	0.9	1.10 (1.05 to 1.15)	<0.001	1.10 (1.05 to 1.15)	<0.001
Non-white ethnicity (1.6%)	40.5 (14.2)	41.5 (18.0)	37.5 (15.2)	0.97 (0.93 to 1.02)	0.2	0.93 (0.89 to 0.98)	0.003	0.91 (0.86 to 0.95)	<0.001
English as second language (0.6%)	20.7 (8.3)	22.2 (7.7)	20.0 (7.5)	1.09 (1.03 to 1.15)	0.003	0.90 (0.85 to 0.95)	<0.001	0.97 (0.92 to 1.03)	0.3
Age >21 years (0%)	18.6 (6.2)	21.0 (10.4)	21.3 (10.4)	1.28 (1.21 to 1.36)	<0.001	1.00 (0.95 to 1.06)	0.9	1.29 (1.21 to 1.37)	<0.001
State school (0.1%)*	63.8 (3.7)	63.9 (10.9)	65.8 (4.8)	1.06 (1.02 to 1.11)	0.008	1.08 (1.03 to 1.13)	0.002	1.15 (1.09 to 1.21)	<0.001
Low socioeconomic status† (28.0%)	5.7 (2.2)	5.5 (1.7)	5.6 (2.5)	0.95 (0.84 to 1.06)	0.3	1.06 (0.94 to 1.19)	0.3	1.00 (0.89 to 1.13)	0.9
Low academic attainment‡ (29.9%)	45.6 (12.9)	36.7 (12.1)	42.8 (9.0)	0.55 (0.52 to 0.58)	<0.001	1.41 (1.33 to 1.48)	<0.001	0.78 (0.74 to 0.82)	<0.001

Proportions are percentage of number of total number of candidates for which data are available.

*Reported attendance (excludes state funded grammar schools).

†Non-professional socioeconomic background according to parental occupation; Office for National Statistics classification system used by Universities and Colleges Admissions Service.

‡Excludes resits. Equivalent to ABB or below at A level or equivalent.

The dichotomisation of sociodemographic and educational variables was guided by previous research on widening participation23 and informed by an initial univariate exploratory analysis of the current data. For example, it has been shown that students from state grammar school do not have poorer performance on the UKCAT16 or in school or university academic attainment24 and are therefore unlikely to be relatively disadvantaged when applying for medical school. This was supported by the initial univariate exploratory analyses in the present dataset. For this reason when creating sociodemographic categories for school type, we placed private and grammar schools in the same (advantaged) category. Previous research has reported an over-representation of people of Asian ethnicity among UK medical students.21 Self reported ethnicity is recorded by UCAS, but this information is not forwarded to universities. When categorising ethnicity in the present study only those describing themselves as of white ethnic background had a significantly higher odds ratio for receiving an offer of a place compared with applicants from other ethnic groups. Consequently we categorised those reporting their ethnicity as non-white as disadvantaged. The “low” academic attainment category was created by identifying those with a standardised tariff score below the mean (that is, <0). The median UCAS tariff in this low attainment group was 320 points, equivalent to between grades ABB and AAB at A level—that is, just below the entry requirement for most UK medical schools (the median tariff for the group as a whole was around 350 points). We defined a low UKCAT score as having a z score of 0 or below—that is, equal to or less than the mean for the peer group. We derived these latter two binary dummy variables to simplify interpretation of the univariate analysis results.

We created a continuous metric of academic performance that included Irish and Scottish qualifications as well as A levels. This was done by expressing the examination results as a percentage of the theoretical maximum tariff scores that could be obtained. Standardised z scores were then derived within peer groups for each nationality; we compared all those taking Scottish “higher” qualifications against each other. In creating this metric we included only the first three highest grades, excluding general studies (or best of five or six in the case of Scottish highers and Irish leaving certificate, respectively) as this is the approach typically used by medical schools. Where examinations had been resat, we retained the grade at first sitting only. We also excluded data from candidates who did not have the minimum number of advanced qualifications required (for example, fewer than five Scottish higher passes). The UKCAT database records reported socioeconomic status using a simplified version of the socioeconomic classification system used by the National Office for Statistics.25 As in previous research on widening participation we classified those who gave themselves a socioeconomic classification rating of four or more as being from a non-professional background. We classified those learning English after the age of 2 years as having English as a second language.

Data analysis

For the data relating to medical school offers, we carried out univariate analyses to estimate the unadjusted (raw) odds ratios for being offered a place after an application to a medical school, given that an applicant was either in or not in a particular sociodemographic category. To control for the dependency of observations nested within the same applicant, we took a multilevel modelling approach with a random intercept for each participant introduced into the logistic regression equation. In the offer related data, however, we could not simultaneously control for the additional potential dependency of observations within the same institution using a multilevel approach. This was because the data did not have a strictly hierarchical structure—that is, different candidates applied to one to four different but overlapping medical schools. All categories were found to be associated with a reduced odds ratio on univariate analysis for at least one type of medical school and offer and so we designated these as “widening participation” groups (table 2) and subsequently entered them into the multiple logistic regression model. After this, we created a multilevel multiple logistic regression model for each type of medical school according to their use of the UKCAT (borderline, factor, or threshold), utilising the eight educational and sociodemographic independent variables available.

Table 2

 Average performance of applicants on UK clinical aptitude test (UKCAT) and advanced educational qualifications (according to total UCAS (Universities and Colleges Admissions Service) tariff score) by group of medical school applied to according to UKCAT use

UKCAT and academic qualification performance	UKCAT in borderline cases (6 medical schools, 10 086 applicants)	UKCAT as factor (9 medical schools, 11 584 applicants)	UKCAT as threshold (7 medical schools, 10 208 applicants)	Regression coefficient (95% CI): factor v borderline use	P value	Regression coefficient (95% CI): threshold v factor use	P value	Regression coefficient (95% CI): threshold v borderline use	P value
UKCAT total score	2524 (197)	2539 (203)	2548 (199)	14.30 (9.91 to 18.69)	<0.001	9.08 (4.60 to 54.02)	<0.001	23.38 (18.67 to 28.09)	<0.001
UCAS tariff:
Total score*	379 (96)	391 (119)	383 (103)	11.63 (8.90 to 14.37)	<0.001	−11.63 (−14.37 to −8.90)	<0.001	4.44 (1.43 to 7.31)	0.004
Standardised z score†	−0.04 (0.9)	0.14 (0.9)	0.01 (0.9)	0.20 (0.18 to 0.23)	<0.001	−0.14 (−0.17 to −0.12)	<0.001	0.06 (0.03 to 0.08)	<0.001

Intergroup differences compared using linear regression, with group as factor variable, for UKCAT and UCAS tariff score. A tobit regression was used to compare standardised UCAS tariff owing to censored nature of measure (maximum z score achievable around 1.27 therefore performance exceeding this could not be observed).

*Counts all valid examination grades, not just “best of three” or equivalent. Does not include grades from general studies from examination resits.

†Generated from “best of three” (or equivalent) grades at advanced level examinations. Standardised separately within 2009 medical application cohort for England/Wales, Scotland and Ireland. For example, a Scottish applicant with a z score of zero is average compared with the tariff attainment of other Scottish candidates for that year.

These three models were built up in a stepwise fashion. Initially we entered the variable that predicted the outcome most strongly. This was followed by the variable that resulted in the largest improvement in fit, until all independent variables had been included in the model. The potential for predictor variables to interact was evaluated by entering each of the potential 28 interaction terms in turn to the logistic regression. In the final models we included interaction terms significant at the P<0.05 level. Improvements in fit were estimated according to the χ2 value relating to a Wald test. In the case of the initial choice of variable, the comparison was with the null model. Model nesting was ensured by including only observations with no missing data in this stage of analysis. To directly compare the three categories of medical school, we pooled the data relating to all the applications without missing values and developed a multilevel multiple logistic model. We entered this time category of UKCAT use as a factor variable with three levels (borderline, factor, and threshold). The association between probability of offer and group of medical school applied to was evaluated by assessing for interactions between this factor variable and the sociodemographic predictors. We explored all combinations of interactions with factor variables by changing the group of medical schools used as the baseline category.

To evaluate the extent to which trends in offer related behaviour were likely to influence the long term make up of the medical work force, we analysed data relating to the final uptake of places at each medical school according to widening participation status. These data had a less complex structure in that there was only one entrant per institution. We carried out a univariate logistic regression to estimate the odds of an entrant to each category of medical school being from a widening participation background. A multilevel multiple regression was also done to control for the effects of potential confounding variables. To control and explore the impact of individual institutional effects (in contrast to membership of a UKCAT usage group), we introduced a random intercept term for each of the 22 medical schools.

Most initial offers made to applicants are conditional. The two main supervening factors between a conditional offer being made and the candidate entering a course are whether the applicant accepts the offer and whether (in the case of a conditional offer) the appropriate grades are obtained at A level or equivalent. Therefore we used multilevel multiple logistic regression analysis to estimate what the predictors of fulfilling the academic requirements were for those applicants holding conditional offers (the outcome being a conditional offer converted to an unconditional offer). A random intercept was introduced to control for the dependency for observations within individual candidates and we tested for all combinations of interactions between independent variables. The additional potential dependency of observations within medical schools could not be controlled for in this model.

We evaluated the relations between missing data status and other variables using logistic regression. Additionally, to estimate the extent to which missing data were missing at random (that is, missing values related to observed variables), we carried out a series of sensitivity analyses using multiple imputation procedures.26 The process involved generating two sets of 10 datasets, where values were imputed for the missing socioeconomic status and advanced qualification values. The imputed values were conditioned on the observed variables. We then repeated the analyses of the offer data using the imputed data to evaluate the extent to which the missing values may have influenced the results. Full details of this procedure are available from the lead author’s website (www.dur.ac.uk/p.a.tiffin/fps).

Results

Descriptive statistics: applications

Overall, the percentages of each category of candidate (where the specific sociodemographic variable was available) were: 44.8% male sex, 39.8% self reported ethnicity as non-white, 21.1% English as a second language, 22.8% aged over 20 years at the time of application, 64.5% attended a non-fee paying or non-grammar school, 5.4% reported being from a non-professional social background, and 40.5% reported academic attainment below the level of AAB at A level (or equivalent). Table 1 shows the proportion of each category of candidate applying to the different categories of medical school along with the standard deviation of the percentages across individual institutions within each group. There was a number of modest, albeit statistically significant, differences between the proportions of widening participation candidates applying to each category of medical school (table 1). The most noticeable difference was between medical schools that used the UKCAT score in borderline cases and those that used it as a factor in admissions, with candidates who achieved relatively low grades at A level or equivalent being more likely to apply to the institutions that used UKCAT as a factor. In general, the variation between proportions of applications to institutions in the same category of UKCAT usage was modest, at around 10% or less, with the exception of ethnicity, which showed more variation within groups.

The overall proportion of missing data for each sociodemographic variable did not significantly vary between groups (table 1), with the exception of the data on advanced qualifications. Such data were missing for 26% of applicants to medical schools using UKCAT in borderline cases, 29% using UKCAT as a factor, and 28% using UKCAT as a threshold (P<0.001 in all cases).

At the time of application, candidates were aware at least of basic admissions policies in relation to the UKCAT and also knew their own test scores. We therefore anticipated that applications to universities with more robust use of the UKCAT would, on average, be made by candidates with higher test scores. Table 2 shows the average performances of candidates applying to each of the three groups of medical schools and advanced level qualifications. Again, there were modest absolute but statistically significant differences between the performances of candidates associated with applications to the three types of institution. In general, candidates applying to medical schools that used UKCAT as a threshold tended to achieve higher UKCAT scores compared with candidates applying to the other two groups of medical schools. Candidates applying to medical schools using UKCAT as a factor tended to go on to obtain higher grades at advanced qualifications, and this was the case whether only the “best of three” A level (or equivalent) grades were counted or whether a total UCAS tariff score was calculated from grades of all examinations taken (with the exception of resits and general studies).

Univariate analysis: applications

For the statistical modelling we utilised observations with complete data only on all variables (fig 1). Tables 3 and 4 depict, respectively, the results of multilevel logistic univariate regressions for the probability of conditional and unconditional offers. Even after controlling for candidate level effects, the unadjusted odds of receiving a conditional offer varied significantly between all three groups of medical schools for most widening participation categories; age and sex being the only exceptions (table 3). The intraclass correlation in this case represents the residual of the latent response variable of each individual candidate; each candidate is conceptualised as having a certain level of a latent trait (an unobserved normally distributed variable). The level of this trait would be manifest in the likelihood of the candidate receiving an offer, thus this trait could be conceptualised as “offerability.” In a multiple logistic regression, the intraclass correlation is the correlation within each individual candidate, once fixed effects are controlled for, such as academic attainment. Thus intraclass correlations of 0.3 to 0.5 (table 3) would be consistent with moderate individual level effects for candidates (as opposed to the effect of group membership of the medical schools applied to). As the odds ratios are raw, such individual level effects would include UKCAT and predicted or actual A level (or equivalent) performance. On average the intraclass correlations were highest for medical schools using UKCAT as a threshold and lowest for those using UKCAT in borderline cases, with all between group differences being significant at the P<0.05 level according to a Wilcoxon test (table 3). This would suggest that individual candidate effects and performance are more predictive of the provision of a conditional offer in those institutions with a stronger use of the UKCAT. A similar pattern was observed for the raw odds of obtaining an unconditional offer, with the association between applicant age and sex weaker than for the other variables. No significant association was observed with applicant age once only unconditional offers were considered. The intraclass correlations were then smaller in magnitude, reflecting weaker candidate level effects. The intraclass correlations for medical schools using UKCAT as a factor were, on average, significantly larger than for the other medical school groups (P<0.001), reflecting the increased predictive power of individual compared with group or unmodelled effects. A plausible reason for this may be a greater emphasis, for example, on A level performance when providing an offer, as some medical schools may insist on the original conditions of an offer being met, whereas other may allow a certain latitude, especially if the number of applicants accepting offers falls short of expectation.

Table 3

 Odds of an application resulting in a conditional offer of a place according to widening participation sociodemographic group and type of medical school’s use of UK clinical aptitude test (UKCAT)

Widening participation group	UKCAT in borderline cases (6 medical schools, 7558 applicants*)				UKCAT as factor (9 medical schools, 8163 applicants*)				UKCAT as threshold (7 medical schools, 6564 applicants*)
	Odds ratio (95% CI)	P value	ICC†		Odds ratio (95% CI)	P value	ICC†		Odds ratio (95% CI)	P value	ICC†
Sex (male v female)	0.70 (0.59 to 0.83)	<0.001	0.34		0.86 (0.74 to 1.00)	0.06	0.38		1.14 (0.96 to 1.35)	0.1	0.49
Ethnicity (non-white v white)	0.46 (0.38 to 0.56)	<0.001	0.33		0.70 (0.59 to 0.82)	<0.001	0.38		0.55 (0.46 to 0.67)	<0.001	0.48
English as second language	0.56 (0.44 to 0.72)	<0.001	0.34		0.55 (0.44 to 0.69)	<0.001	0.38		0.50 (0.34 to 0.59)	<0.001	0.48
Age (>21 years at application)	0.77 (0.44 to 1.35)	0.4	0.35		0.50 (0.34 to 0.73)	<0.001	0.37		0.34 (0.21 to 0.56)	<0.001	0.49
School type (state v private or grammar)	0.58 (0.49 to 0.68)	<0.001	0.34		0.34 (0.29 to 0.39)	<0.001	0.34		0.62 (0.52 to 0.74)	<0.001	0.49
Socioeconomic class 4 or 5‡	0.31 (0.19 to 0.50)	<0.001	0.34		0.51 (0.40 to 0.65)	<0.001	0.38		0.50 (0.33 to 0.76)	0.001	0.49
Below average academic attainment	0.27 (0.22 to 0.32)	<0.001	0.29		0.37 (0.25 to 0.56)	<0.001	0.30		0.18 (0.14 to 0.22)	<0.001	0.45
Below average UKCAT total score	0.42 (0.35 to 0.51)	<0.001	0.34		0.18 (0.15 to 0.22)	<0.001	0.49		0.04 (0.03 to 0.05)	<0.001	0.30

ICC=intraclass correlation. A random intercept was allowed for each candidate.

*Applicants with complete data.

†ICC represents residual correlation of latent responses within each applicant.

‡According to Office for National Statistics classification system.

Table 4

 Unadjusted (raw) odds of an application resulting in an unconditional offer of a place according to widening participation sociodemographic group and use of UKCAT by medical school

Widening participation group	UKCAT in borderline cases (6 medical schools, 7690 applicants*)			UKCAT as factor (9 medical schools, 8692 applicants*)			UKCAT as threshold (7 medical schools, 6725 applicants*)
	Odds ratio (95% CI)	P value	ICC†	Odds ratio (95% CI)	P value	ICC†	Odds ratio (95% CI)	P value	ICC†
Sex (male v female)	0.79 (0.69 to 0.92)	0.002	0.06	0.90 (0.79 to 1.03)	0.1	0.22	1.15 (1.01 to 1.32)	0.04	0.14
Ethnicity (non-white v white)	0.53 (0.45 to 0.62)	<0.001	0.05	0.55 (0.47 to 0.64)	<0.001	0.21	0.69 (0.59 to 0.80)	<0.001	0.12
English as second language	0.67 (0.54 to 0.84)	<0.001	0.06	0.54 (0.44 to 0.66)	<0.001	0.21	0.55 (0.44 to 0.69)	<0.001	0.13
Age (>21 years at application)	0.93 (0.58 to 1.50)	0.8	0.07	0.77 (0.56 to 1.05)	0.1	0.22	0.74 (0.51 to 1.06)	0.1	0.13
School type (state v private or grammar)‡	0.62 (0.53 to 0.71)	<0.001	0.05	0.58 (0.51 to 0.67)	<0.001	0.21	0.73 (0.64 to 0.84)	<0.001	0.13
Socioeconomic class 4 or 5§	0.41 (0.27 to 0.63)	<0.001	0.06	0.48 (0.34 to 0.70)	<0.001	0.21	0.69 (0.49 to 0.96)	0.03	0.13
Below average academic attainment¶	0.27 (0.23 to 0.31)	<0.001	<0.01	0.18 (0.15 to 0.22)	<0.001	0.15	0.27 (0.23 to 0.31)	<0.001	0.03
Below average UKCAT total score**	0.50 (0.43 to 0.58)	<0.001	0.05	0.28 (0.24 to 0.33)	<0.001	0.16	0.09 (0.08 to 0.12)	<0.001	<0.001

ICC=intraclass correlation. A random intercept was allowed for each candidate.

*Applicants with complete data.

†Represents residual correlation of latent responses within each applicant.

‡Reported attendance at a state school, excludes state funded grammar schools.

§ Non-professional socioeconomic background according to parental occupation; Office for National Statistics classification system used by Universities and Colleges Admissions Service.

¶Low academic attainment (excluding resits) equivalent to ABB or below at A level or equivalent.

**Total UKCAT z score of zero or below.

Multilevel multiple logistic regression: applications

Tables 5 and 6 show, respectively, the results for the separate multilevel multiple logistic regressions for the probability of conditional and unconditional offers. The results in table 5 depict three varying models for the prediction of a conditional offer. For those applications relating to medical schools using UKCAT as borderline, six independent and statistically significant predictor variables existed, including sex, socioeconomic status, ethnicity, type of school attended, and academic attainment, with academic achievement being most strongly predictive of an offer. The UKCAT score was a relatively weak independent predictor in applications relating to this group of medical schools in that the odds of receiving a conditional offer were only increased by roughly 20% for every standard deviation of standardised score above the mean that was scored (odds ratio 1.23, 95% confidence interval 1.11 to 1.37). The model for applications to medical schools that used UKCAT in borderline cases had three statistically significant interaction terms; age and academic attainment, sex and school type attended, and UKCAT score and English as a second language. The first of these indicates that for candidates aged more than 20 at the time of application, A level (or equivalent) attainment conferred relatively less advantage when a conditional offer was made compared with younger applicants. This is highly plausible given that many older applicants will be applying on the basis of a university degree rather than on A level attainment. The interaction term between male sex and school type attended implies that the relative disadvantage of having attended a state school was reduced for men compared with women. Lastly, the interaction between UKCAT score and English as a second language suggests that an increased UKCAT score conferred a greater advantage for those for whom English was a second language compared with native English speakers. Medical schools using the UKCAT score as a factor in the admissions process had five significant and independent predictors of the probability of a conditional offer. In contrast with the medical schools using UKCAT score in borderline cases, the strongest of these was standardised UKCAT score, with an increase in 1 SD above the mean more than doubling the odds of an offer, all other factors being equal (odds ratio 2.31, 95% confidence interval 2.07 to 2.58). As in the model with medical schools that used the UKCAT score in borderline cases, interactions were significant between academic attainment and age and UKCAT score and English as a second language. In addition the interaction term for academic attainment and UKCAT score was significant, indicating some degree of synergy between A level (or equivalent) grades and UKCAT score when predicting the probability of a conditional offer. For medical schools with UKCAT used as a threshold score in the admissions process the only significant independent predictors were UKCAT score (odds ratio 8.59, 95% confidence interval 6.96 to 10.62) and, to a lesser extent, academic attainment (1.63, 1.46 to 1.82). As with the other two categories of university the interaction between UKCAT score and English as second language was significant. Two other interaction terms were also significant. Firstly, the interaction between sex and UKCAT score, suggesting that for males an increase in UKCAT score conferred less of an advantage than for females when seeking a conditional place offer. This is interesting given that, on average, men outperform women on the UKCAT.16 Secondly, the interaction between socioeconomic status and English as a second language was significant and suggested that those from a lower socioeconomic background who were not native English speakers were relatively disadvantaged compared with candidates who did not belong to this subgroup.

Table 5

 Results of multilevel multiple logistic regression analyses with outcome variable a conditional offer of a place at medical school

Predictor variables	Odds ratio (95% CI) for receiving offer	P value	ICC*
UKCAT scores for borderline cases (n=7558)†:
Academic attainment (standardised tariff score in SDs)	2.63 (2.27 to 3.05)	<0.001	0.23
Non-white ethnicity	0.60 (0.49 to 0.73)	<0.001
Male sex	0.55 (0.43 to 0.70)	<0.001
Attendance at state (non-grammar) school	0.62 (0.51 to 0.76)	<0.001
Standardised UKCAT total score (SDs)	1.23 (1.11 to 1.37)	<0.001
Socioeconomic class 4 or 5‡	0.51 (0.32 to 0.81)	0.004
Age >21 years	1.64 (0.87 to 3.06)	0.1
English as second language	1.04 (0.79 to 1.36)	0.8	—
Academic attainment-age interaction	0.53 (0.36 to 0.77)	0.001	—
Male-school type interaction	1.57 (1.14 to 2.16)	0.005	—
UKCAT score-English as second language interaction	1.36 (1.01 to 1.84)	0.04	—
UKCAT scores as factor for admissions (n=8163)†:
Standardised UKCAT total test score (SDs)	2.31 (2.07 to 2.58)	<0.001	0.12
Academic attainment (standardised tariff score in SDs)	2.27 (1.96 to 2.62)	<0.001
Attendance at state (non-grammar) school	0.52 (0.45 to 0.60)	<0.001
Male	0.78 (0.68 to 0.89)	<0.001
Socioeconomic class 4 or 5‡	0.62 (0.42 to 0.91)	0.01
English as second language	0.79 (0.61 to 1.03)	0.08
Age >21 years	1.17 (0.77 to 1.78)	0.5	—
Non-white ethnicity	0.95 (0.81 to 1.12)	0.5	—
Academic attainment-age interaction	0.42 (0.33 to 0.52)	<0.001	—
Academic attainment-UKCAT score interaction	1.17 (1.05 to 1.29)	0.003	—
UKCAT score-English as second language interaction	1.54 (1.14 to 2.07)	0.005	—
UKCAT scores as threshold (n=6564)†:
Standardised UKCAT total score (SDs)	8.59 (6.96 to 10.62)	<0.001	0.13
Academic attainment (standardised tariff score in SDs)	1.63 (1.46 to 1.82)	<0.001
Socioeconomic class 4 or 5‡	1.27 (0.84 to 1.91)	0.3
English as second language	1.16 (0.87 to 1.55)	0.3
Non-white ethnicity	1.06 (0.88 to 1.28)	0.5	—
Attendance at state (non-grammar) school	1.02 (0.88 1.18)	0.8	—
Male sex	1.02 (0.85 to 1.23)	0.8	—
Age >21 years	1.03 (0.65 to 1.64)	0.9	—
Male-UKCAT score interaction	0.68 (0.53 to 0.84)	0.001	—
UKCAT score-English as second language interaction	0.68 (0.49 to 0.94)	0.02	—
Socioeconomic class 4 or 5-English as second language interaction	0.36 (0.15 to 0.86)	0.02	—

ICC=intraclass correlation. Average cluster size (UKCAT consortium applications per candidate): 1.4 for medical schools using UKCAT score in borderline cases, 1.5 for those using UKCAT as factor in admissions, and 1.3 for those using UKCAT as threshold score.

*Represents residual correlation of latent responses for each applicant.

†Number of applications, rather than number of candidates.

‡Office for National Statistics socioeconomic classification system.

Table 6

 Results of multilevel multiple logistic regression analyses with outcome variable an unconditional offer of a place at medical school

Predictor variables	Odds ratio (95% CI) for receiving offer	P value	ICC*
UKCAT scores for borderline cases (n=7690)†:
Academic attainment (standardised tariff score in SDs)	2.78 (2.41 to 3.20)	<0.001	<0.001
Non-white ethnicity	0.63 (0.52 to 0.76)	<0.001
Attendance at state (non-grammar) school	0.64 (0.53 to 0.76)	<0.001
Male sex	0.65 (0.53 to 0.81)	<0.001
Standardised UKCAT total score (SDs)	1.22 (1.09 to 1.36)	<0.001
Socioeconomic class 4 or 5‡	0.61 (0.39 to 0.95)	0.03
Age >21 years	1.70 (0.98 to 2.93)	0.06
English as second language	1.09 (0.85 to 1.40)	0.5	—
Academic attainment-age interaction	0.49 (0.35 to 0.68)	<0.001	—
Academic attainment-UKCAT score interaction	0.79 (0.69 to 0.90)	0.001	—
Male-school type interaction	1.45 (1.09 to 1.94)	0.01	—
UKCAT score-English as second language interaction	1.36 (1.03 to 1.79)	0.03	—
UKCAT scores as factor in admissions (n=8692)†:
Standardised UKCAT total score (SDs)	2.00 (1.80 to 2.21)	<0.001	0.1
Academic attainment (standardised tariff score in SDs)	2.28 (1.96 to 2.64)	<0.001
Non-white ethnicity	0.64 (0.53 to 0.76)	<0.001
Attendance at state (non-grammar) school	0.75 (0.66 to 0.85)	<0.001
Male	0.85 (0.74 to 0.96)	0.01
English as second language	0.46 (0.25 to 0.84)	0.01
Socioeconomic class 4 or 5‡	0.72 (0.50 to 1.04)	0.08
Age >21 years	1.35 (0.93 to 1.97)	0.1
Academic attainment-age interaction	0.48 (0.39 to 0.60)	<0.001	—
Academic attainment-UKCAT score interaction	0.75 (0.67 to 0.84)	<0.001	—
UKCAT score-English as second language interaction	1.46 (1.11 to 1.93)	0.007	—
Ethnicity-English as second language interaction	2.10 (1.09 to 4.05)	0.03	—
UKCAT score as threshold (n=6725)†:
Standardised UKCAT total score (SDs	3.37 (3.03 to 3.74)	<0.001	<0.001
Academic attainment (standardised tariff score in SDs)	2.07 (1.77 to 2.41)	<0.001
Age >21 years	1.53 (0.99 to 2.38)	0.06
English as second language	0.86 (0.66 to 1.10)	0.2
Non-white ethnicity	1.07 (0.90 to 1.28)	0.5
Male sex	0.95 (0.83 to 1.09)	0.5	—
Attendance at state (non-grammar) school	0.97 (0.85 to 1.11)	0.7	—
Socioeconomic class 4 or 5‡	1.04 (0.74 to 1.46)	0.8	—
Academic attainment-UKCAT score interaction	0.81 (0.71 to 0.92)	0.001	—
Academic attainment-age interaction	0.72 (0.56 to 0.94)	0.02	—

ICC=intraclass correlation. Average cluster size (UKCAT consortium applications per candidate): 1.4 for medical schools using borderline UKCAT score, 1.6 for those using UKCAT score as a factor, and 1.3 for those using UKCAT score as a threshold.

*Represents residual correlation of latent responses for observations within each applicant.

†Number of applications rather than number of candidates.

‡Office for National Statistics socioeconomic classification.

The results in table 6 depict the three models for the prediction of an unconditional offer, according to category of medical school. The results were broadly similar to those in table 5, according to the probability of conditional place offers. Medical school that used UKCAT scores in borderline cases and as a factor had six independent and statistically significant principal predictor variables: sex, socioeconomic status, ethnicity, type of school attended, UKCAT score, and academic attainment. For medical schools using UKCAT as a threshold, the only sociodemographic variables that were independent and significant predictors of an offer were UKCAT score and academic attainment. In addition there was a trend of borderline statistical significance for older candidates to more likely receive an unconditional offer than those under 20 at application (odds ratio 1.53, 95% confidence interval 0.99 to 2.38, P=0.06). This observation is unsurprising as many older candidates have already obtained the required qualifications for entry (although specific graduate entry courses were excluded from the analysis). Each model also included between two and four statistically significant interaction terms. In addition to the interaction terms that were significant in the results outlined in table 5 the model for medical schools using UKCAT as a factor relating to the probability of an unconditional model also showed a significant interaction between ethnicity and English as a second language (2.10, 1.09 to 4.05, P=0.03). This implied that those who reported their ethnicity as non-white and were not native English speakers were relatively less disadvantaged compared with white non-native English speakers when obtaining an unconditional offer. This interaction term seemed to be the consequence of a small number of white non-native English speaking candidates, relatively few of whom obtained unconditional offers.

Tables 7 and 8 present the results of the combined multilevel multiple logistic regression. The models contained interaction terms concerning medical school group (entered as a factor variable), where the baseline category was varied. Therefore it was necessary to depict the results for the logistic regression models where medical schools using UKCAT as a threshold (table 7) and using it as a factor (table 8) had been used as the base categories separately. Consequently the odds ratios and P values for all statistically significant combinations of interaction terms are presented as pairwise comparisons (for example, borderline group versus threshold group by age interaction) rather than overall interaction effects. This seemed more appropriate given that UKCAT usage was a factor variable with more than two levels but not ordinal (ordered categorical) in nature—that is, failing to fulfil the “proportional odds” assumption. The odds of an applicant receiving a conditional offer did not significantly vary between the different categories of universities once the effects of other predictors and interaction terms were controlled for (tables 7 and 8). However, this was not true for unconditional offers, where it was less likely that a candidate applying to a medical school that used UKCAT as a threshold received an unconditional offer compared with the other two categories of medical school, implying that, on average, relatively fewer applicants to the medical schools using UKCAT as a threshold were satisfying the academic conditions set as part of their conditional offers. The interaction terms involving the group of medical school applied to largely mirrored the findings from the separate models (tables 5 and 6); the interaction terms highlighted intergroup differences in relation to UKCAT score, academic attainment, ethnicity, age, school type attended, and socioeconomic status. Interaction terms involving group and English as second language were not statistically significant at the P<0.05 level for either conditional or unconditional offers. Likewise, significant interactions between sex and type of university applied to were not observed, although an interaction between an application to a medical school that used UKCAT scores in borderline cases (versus those using threshold scores) and sex was of marginal statistical significance (odds ratio 0.83, 95% confidence interval 0.68 to 1.01, P=0.06). This indicated a trend towards male applicants being slightly less likely to receive an offer from a medical school that used the UKCAT in borderline cases as opposed to in a threshold manner. Not all the intergroup differences were present or in the direction originally hypothesised (tales 7 and 8). For example, interaction terms suggested that an applicant from a state school (in this case excluding grammar schools) was less likely to receive a conditional offer from a medical school that used UKCAT scores in borderline cases compared with one using threshold scores (0.74, 0.60 to 0.90, P=0.004) but were more likely to receive such an offer than from a medical school that used UKCAT as a factor (1.47, 1.21 to 1.80, P<0.001). Nevertheless this finding was consistent with the results from the separate models for the prediction of conditional offer provision (table 5) in that medical schools using UKCAT as a factor in the admissions process showed a relatively high degree of disadvantage for state school students (0.52, 0.45 to 0.60, P<0.001) even after controlling for the effects of other, potentially confounding, variables and interactions. Medical schools that used UKCAT as a threshold were also less likely to provide a conditional offer to an older applicant compared with the other two categories of medical school.

Table 7

 Results of multilevel multiple logistic regression analyses with outcome variable as conditional or unconditional offer of a place to study medicine. Values are odds ratios (95% confidence intervals) unless stated otherwise

Predictor variables	Conditional offer (22 285 applications, 8303 candidates)	P value	Unconditional offer (23 107 applications, 8348 candidates)	P value
Standardised UKCAT total score (SDs)	6.58 (5.86 to 7.61)	<0.001	3.56 (3.22 to 3.94)	<0.001
Academic attainment (standardised tariff score in SDs)	1.85 (1.65 to 2.07)	<0.001	2.34 (2.07 to 2.64)	<0.001
Male sex	0.76 (0.70 to 0.84)	<0.001	0.86 (0.80 to 0.93)	<0.001
Non-white ethnicity	0.99 (0.82 to 1.20)	0.9	1.06 (0.90 to 1.25)	0.5
English as second language	0.95 (0.81 to 1.11)	0.5	0.97 (0.85 to 1.12)	0.5
Age >21 years	0.72 (0.42 to 1.21)	0.2	1.78 (1.39 to 2.28)	<0.001
Attendance at state (non-grammar) school	1.02 (0.87 to 1.18)	0.8	0.97 (0.85 to 1.12)	0.7
Socioeconomic class 4 or 5*	0.98 (0.67 to 1.42)	0.9	0.79 (0.64 to 0.98)	0.03
Medical school group, according to UKCAT use:
In borderline cases	1.16 (0.96 to 1.40)	0.1	1.36 (1.13 to 1.64)	0.001
As factor	1.11 (0.91 to 1.34)	0.3	1.44 (1.21 to 1.72)	<0.001
Threshold use (base category)	1	NA	1	NA
UKCAT score-borderline use interaction	0.18 (0.16 to 0.22)	<0.001	0.37 (0.32 to 0.42)	<0.001
UKCAT score-factor use interaction	0.39 (0.34 to 0.45)	<0.001	0.59 (0.52 to 0.67)	<0.001
Academic achievement-borderline use interaction	1.39 (1.18 to 1.63)	<0.001	1.18 (1.01 to 1.39)	0.04
Academic achievement-factor use interaction	1.22 (1.05 to 1.42)	0.01	0.93 (0.81 to 1.06)	0.3
Non-white-borderline use interaction	0.62 (0.49 to 0.79)	<0.001	0.61 (0.49 to 0.77)	<0.001
Non-white- factor use interaction	0.88 (0.71 to 1.10)	0.3	0.65 (0.53 to 0.80)	<0.001
Age >21 years-borderline use interaction	2.21 (1.09 to 4.48)	0.03	NA	NS
Age >21 years-factor use interaction	2.32 (1.29 to 4.18)	0.005	NA	NS
School type-borderline use interaction	0.74 (0.60 to 0.90)	0.004	0.76 (0.62 to 0.92)	0.005
School type-factor use interaction	0.50 (0.41 to 0.61)	<0.001	0.78 (0.65 to 0.93)	0.006
Socioeconomic class*-borderline use interaction	0.52 (0.29 to 0.91)	0.02	NA	NS
Socioeconomic class*-factor use interaction	0.63 (0.37 to 1.06)	0.08	NA	NS
Academic attainment-age interaction	0.52 (0.44 to 0.62)	<0.001	0.46 (0.49 to 0.65)	<0.001
UKCAT score-age interaction	0.67 (0.50 to 0.89)	0.006	0.56 (0.43 to 0.72)	<0.001
UKCAT score-non-white interaction	1.19 (1.04 to 1.36)	0.01	NA	NS
Academic attainment-UKCAT score interaction	NA	NS	0.73 (0.49 to 0.65)	<0.001

NA=not applicable; NS=not significant. Only interaction terms that were significantly predictive of an offer at p<0.05 level are included in each model. Group comprising medical schools using UKCAT as threshold was base category.

*Office for National Statistics socioeconomic classification.

Table 8

 Results of multilevel multiple logistic regression analyses with outcome variable conditional or unconditional offer of a place at medical school. Values are odds ratios (95% confidence intervals) unless stated otherwise

Predictor variable	Conditional offer (n=22 285 applications, 8303 candidates)	P value	Unconditional offer (n=23 107 applications, 8348 candidates	P value
Standardised UKCAT total score (SDs)	2.61 (2.36 to 2.90)	<0.001	2.11 (2.93 to 2.29)	<0.001
Academic attainment (standardised tariff score in SDs)	2.26 (1.99 to 2.55)	<0.001	2.17 (1.94 to 2.42)	<0.001
Male sex	0.76 (0.70 to 0.84)	<0.001	0.86 (0.80 to 0.93)	<0.001
Non-white ethnicity	0.87 (0.73 to 1.04)	0.1	0.69 (0.60 to 0.80)	<0.001
English as second language	0.95 (0.81 to 1.11)	0.5	0.96 (0.83 to 1.10)	0.5
Age >21 years	1.66 (1.10 to 2.49)	0.02	1.78 (1.39 to 2.28)	<0.001
Attendance at state (non-grammar) school	0.51 (0.44 to 0.58)	.<0.001	0.76 (0.67 to 0.85)	<0.001
Socioeconomic class 4 or 5*	0.61 (0.42 to 0.91)	0.01	0.79 (0.64 to 0.98)	0.03
Medical school group, according to UKCAT use:
In borderline cases	1.05 (0.87 to 1.25)	0.6	0.95 (0.80 to 1.12)	0.5
As factor (base category)	1	NA	1	NA
Threshold use	0.90 (0.75 to 1.09)	0.3	0.69 (0.58 to 0.83)	<0.001
UKCAT score-borderline use interaction	0.47 (0.41 to 0.54)	<0.001	0.62 (0.56 to 0.70)	<0.001
UKCAT score-threshold use interaction	2.56 (2.20 to 2.96)	<0.001	1.69 (1.50 to 1.91)	<0.001
Academic achievement-borderline use interaction	1.14 (0.96 to 1.34)	0.1	1.28 (1.10 to 1.49)	0.001
Academic achievement-threshold use interaction	0.82 (0.70 to 0.95)	0.01	1.08 (0.95 to 1.23)	0.3
Non-white-borderline use interaction	0.71 (0.56 to 0.89)	0.003	0.94 (0.75 to 1.17)	0.6
Non-white-threshold use interaction	1.13 (0.91 to 1.42)	0.3	1.53 (1.24 to 1.89)	<0.001
Age-borderline use interaction	0.96 (0.50 to 1.82)	0.9	NA	NS
Age-threshold use interaction	0.43 (0.24 to 0.78)	0.005	NA	NS
School type-borderline use interaction	1.47 (1.21 to 1.80)	<0.001	0.98 (0.81 to 1.17)	0.8
School type-threshold use interaction	2.00 (1.64 to 2.44)	<0.001	1.29 (1.08 to 1.54)	0.006
Socioeconomic class*-borderline use interaction	0.82 (0.46 to 1.47)	0.5	NA	NS
Socioeconomic class*-threshold use interaction	1.59 (0.49 to 2.69)	0.08	NA	NS
Academic attainment-UKCAT score interaction	NA	NS	0.73 (0.68 to 0.79)	<0.001
UKCAT score-age interaction	0.67 (0.50 to 0.89)	0.006	0.56 (0.43 to 0.72)	<0.001
UKCAT score-non-white interaction	1.19 (1.04 to 1.36)	0.01	NA	NS
Academic attainment-age interaction	0.52 (0.44 to 0.62)	<0.001	0.56 (0.49 to 0.65)	<0.001

NA=not applicable; NS=not significant. Only interaction terms that were significantly predictive of an offer at P<0.05 level are included in each respective model. Group comprising medical schools using UKCAT as factor was base category.

*Office for National Statistics socioeconomic classification.

Descriptive statistics and univariate analysis: medical school entrants

In total, data related to 4456 medical school entrants to the 22 universities analysed. Table 9 and figure 2 depict the breakdown of the proportion of each medical school intake by widening participation status. Some categories of institutions appeared fairly homogenous in relation to the proportion of candidates of widening participation status taking up places, whereas others were relatively diverse. For example, medical schools using UKCAT as a threshold consistently admitted more than around 55% of their medical students from state schools, whereas more variation was observed in medical schools using UKCAT as a factor in the admissions process (fig 2).

Table 9

 Sociodemographic and educational characteristics of medical school entrants according to type of medical school (use of UK clinical aptitude test (UKCAT)) to which they were admitted. Values are mean (SD) percentage of entrants from widening participation group unless stated otherwise

Widening participation group (% missing)	UKCAT in borderline cases (6 medical school, 1027 entrants)	UKCAT as factor (9 medical schools, 1793 entrants)	UKCAT as threshold (7 medical schools, 1636 entrants)	Odds ratio (95% CI): factor v borderline use	P value	Odds ratio (95% CI): threshold v factor use	P value	Odds ratio (95% CI): threshold v borderline use	P value
Males (0%)	38.0 (6.6)	41.2 (3.7)	50.2 (4.9)	1.12 (0.96 to 1.31)	0.2	1.39 (1.21 to 1.59)	<0.001	1.55 (1.33 to 1.82)	<0.001
Non-white (1.4%)	31.0 (12.5)	27.6 (16.9)	32.3 (13.9)	0.98 (0.83 to 1.16)	0.8	1.13 (0.98 to 1.31)	0.1	1.11 (0.93 to 1.32)	0.2
English as second language (0.8%)	14.8 (4.4)	12.7 (7.4)	12.4 (4.8)	0.86 (0.69 to 1.08)	0.2	0.93 (0.75 to 1.14)	0.5	0.80 (0.63 to 1.01)	0.06
Age >21 years (0%)	11.4 (7.9)	17.4 (12.2)	18.4 (10.7)	1.62 (1.28 to 2.04)	<0.001	1.03 (0.86 to 1.23)	0.7	1.67 (1.32 to 2.11)	<0.001
State school (0%)*	53.5 (5.6)	53.9 (13.1)	61.2 (6.1)	0.90 (0.77 to 1.05)	0.2	1.46 (1.28 to 1.68)	<0.001	1.31 (1.12 to 1.54)	0.001
Low socioeconomic class† (26.3%)	2.8 (0.7)	3.4 (1.7)	4.4 (2.4)	1.19 (0.70 to 2.02)	0.5	1.28 (0.85 to 1.94)	0.2	1.51 (0.90 to 2.53)	0.1
Low academic attainment‡ (20.8%)	29.3 (8.0)	14.9 (12.9)	22.6 (14.6)	0.35 (0.28 to 0.44)	<0.001	2.23 (1.85 to 2.79)	<0.001	0.81 (0.66 to 0.98)	0.03

Proportions are percentage of number of total number of candidates for which data are available.

*Excludes state funded grammar schools.

†Non-professional socioeconomic background according to parental occupation; Office for National Statistics classification system used by Universities and Colleges Admissions Service.

‡Excludes resits, equivalent to ABB or below at A level or equivalent.

Fig 2 Percentage of medical school entrants who were male, reported non-white ethnicity, reported speaking English as a second language, were aged over 21 years at application, had not attended an independent or grammar school, reported a non-professional socioeconomic background (according to socioeconomic classification system of National Office for Statistics)25, and who obtained relatively low UCAS tariffs (equating to grades ABB or below at A level examinations, or Scottish or Irish equivalents) for each of 22 participating institutions, grouped by use of UKCAT in admissions process. UCAS=Universities and Colleges Admissions Service; UKCAT=UK clinical aptitude test

The proportion of missing data did not vary significantly between groups (table 9), with the exception of advanced qualifications; this variable was missing for 26% of entrants to medical schools that used UKCAT scores in borderline cases, 29% for those that used UKCAT as a factor in the admissions process, and 28% of those that used UKCAT as a threshold, with the difference between factor and threshold groups and the borderline group being significant (P<0.01 in all cases).

Table 10 illustrates the relative performance of entrants on the UKCAT and at A level examinations (or equivalent), the latter in terms of UCAS tariff. In table 10 the UCAS tariff is presented in two ways; as absolute points from all advanced level examinations (excepting resits and general studies), and as a standardised z score calculated on the basis of the “best of three” (or equivalent number) of advanced level grades. Those entrants to medical schools that used the UKCAT as a threshold had significantly higher total scores, on average, than those entrants to medical schools that used UKCAT as a factor, who, in turn, had on average higher scores than entrants to medical schools that used the UKCAT in borderline cases (P<0.001 in all cases). Both “uncensored” (no maximum set) and censored (for example, “best of three”) standardised UCAS tariff scores were significantly higher (P<0.05 and P<0.001, respectively) in entrants to medial schools that used UKCAT as a factor compared with the other two categories of medical school.

Table 10

 Average performance of applicants on UK clinical aptitude test (UKCAT) and advanced educational qualifications (according to total Universities and Colleges Admissions Service (UCAS) tariff score) by group of medical schools applied to according to UKCAT use. Values are means (standard deviations) unless stated otherwise

UKCAT and academic qualification performance	UKCAT in borderline cases (6 medical schools, 1027 entrants)	UKCAT as factor (9 medical schools, 1793 entrants)	UKCAT as threshold (7 medical schools, 1636 entrants)	Regression coefficient (95% CI): factor v borderline use	P value	Regression coefficient (95% CI): threshold v factor use	P value	Regression coefficient (95% CI): threshold v borderline use	P value
UKCAT total score	2566 (166)	2638 (171)	2681 (141)	71.28 (58.30 to 84.26)	<0.001	43.45 (34.87 to 54.02)	<0.001	57.36 (51.45 to 63.28)	<0.001
UCAS tariff:
Total score*	403 (79)	413 (113)	404 (93)	9.67 (1.28 to 18.05)	0.02	−8.59 (−16.11 to −1.07)	0.03	1.08 (−7.50 to 9.65)	0.8
Standardised z score†	0.33 (0.5)	0.510 (0.6)	0.37 (0.5)	0.18 (0.12 to 0.25)	<0.001	−0.15 (−0.21 to −0.10)	<0.001	0.03 (−0.04 to 0.10)	0.4

Tobit regression was used to compare standardised UCAS tariff owing to censored nature of measure (maximum z score achievable around 1.27 therefore performance exceeding this could not be observed).

*All valid examination grades, not just “best of three” or equivalent. Does not include grades from general studies from examination resits.

†Generated from “best of three” (or equivalent) grades at advanced level examinations. Standardised separately within 2009 medical application cohort for England/Wales, Scotland and Ireland. For example, a Scottish applicant with a z score of zero is average compared with the tariff attainment of other Scottish candidates for that year.

Multilevel multiple logistic regression: medical school entrants

To ensure nesting, only observations from 2679 entrants with complete data were utilised, out of the total of 4456 individuals (60%). Table 11 shows the results for sex, ethnicity, English as second language, and age. The manner in which UKCAT was used was entered as a three level factor variable. Unless stated otherwise, the group of medical schools that used UKCAT as a factor was used as the baseline category. This simplified the presentation of the results because although all the combinations of interactions were explored, those that were significant (P<0.05) mainly involved this group of medical schools as one of the comparators. Several of the effects observed in the raw univariate single level analysis (table 9) were attenuated or not observed after controlling for individual institutional effects and potential confounding. For example, entrants to medical schools that used UKCAT as a factor or as a threshold continued to be more likely to be male than those admitted to medical schools that used UKCAT in borderline cases. In contrast, the trend for entrants to medical schools that used UKCAT as a factor or threshold to be over 20 years at application was no longer apparent after controlling for individual institutional and the effect other sociodemographic variables. The intraclass correlation (0.22) was larger than for the other widening participation categories depicted in table 11. This would suggest a relatively large institutional level effect compared, for example, with that for the association with an entrant’s sex (intraclass correlation=0.002). This is supported by the descriptive statistics observed in table 9 and figure 2 where the relatively large variation between institutions in the same UKCAT usage category for intake of older students could be contrasted with the relative homogeneity within groups for the proportion of male entrants. These findings would suggest that the trend for mature entrants reflects one or more individual universities taking in either large or small proportions of mature students, rather than an effect associated with type of UKCAT usage (table 9). However, not all the trends observed were attenuated after controlling for institutional effects and potential confounding variables; the difference between the odds of an entrant to a medical school using UKCAT in borderline cases compared with one from those using UKCAT as a factor having English as a second language actually increased in significance (P value from 0.2 to 0.04) once this more complex modelling was performed.

Table 11

 Results of multilevel multiple logistic regression analyses for prediction of sex, ethnicity, English as second language status, and age (>21 years at application) of entrants to 22 participating medical schools with complete data available (n=2679)

Predictors	Odds ratio (95% CI)	P value	ICC
Male entrant:
UKCAT in borderline cases v as factor	0.65 (0.48 to 0.89)	0.006	0.002
UKCAT as threshold v in borderline cases	1.74 (1.25 to 2.41)	0.001
UKCAT as threshold v as factor	1.13 (0.85 to 1.46)	0.4
Standardised UKCAT total score (SDs)	1.30 (1.15 to 1.46)	<0.001
Non-white ethnicity	1.26 (1.02 to 1.55)	0.03
Academic attainment (standardised tariff score in SDs)	0.88 (0.77 to 1.00)	0.05
English as second language	0.87 (0.65 to 1.18)	0.4
Socioeconomic class 4 or 5†	1.06 (0.67 to 1.65)	0.8
Age >21 years	0.96 (0.61 to 1.52)	0.9
Attendance at state (non-grammar) school	1.00 (0.85 to 1.17)	0.9
School type-UKCAT as factor (v in borderline cases) interaction	2.27 (1.51 to 3.41)	<0.001
Non-white entrant:
UKCAT in borderline cases v as factor	0.95 (0.40 to 2.26)	0.9	0.16
UKCAT as threshold v in borderline cases	1.61 (0.65 to 4.01)	0.3
UKCAT as threshold v as factor	1.53 (0.67 to 3.49)	0.3
Standardised UKCAT total score (SDs)	0.65 (0.55 to 0.78)	<0.001
English as second language	18.15 (12.32 to 26.74)	<0.001
Socioeconomic class 4 or 5†	4.96 (2.86 to 8.60)	<0.001
Attendance at state (non-grammar) school	0.64 (0.51 to 0.79)	<0.001
Male sex	1.28 (1.03 to 1.59)	0.03
Age >21 years	0.78 (0.38 to 1.59)	0.5
Academic attainment (standardised tariff score in SDs)	0.97 (0.80 to 1.17)	0.7
UKCAT score-English as second language interaction	2.25 (1.36 to 3.73)	0.002
Entrant with English as second language:
UKCAT in borderline cases v as factor	2.15 (1.03 to 4.48)	0.04	<0.001
UKCAT as threshold v in borderline cases	0.61 (0.27 to 1.24)	0.2
UKCAT as threshold v as factor	1.31 (0.56 to 3.04)	0.5
Non-white ethnicity	28.49 (14.95 to 54.28)	<0.001
Standardised UKCAT total score (SDs)	0.42 (0.27 to 0.65)	<0.001
Attendance at state (non-grammar) school	1.36 (1.00 to 1.83)	0.05
Socioeconomic class 4 or 5†	1.44 (0.82 to 2.53)	0.2
Age >21 years	0.53 (0.19 to 1.48)	0.2
Male sex	0.87 (0.65 to 1.18)	0.4
Academic attainment (standardised tariff score in SDs)	0.94 (0.76 to 1.17)	0.6
UKCAT score-non-white ethnicity interaction	2.09 (1.28 to 3.42)	0.003
UKCAT in borderline cases (v as factor)-non-white ethnicity interaction	0.41 (0.17 to 0.97)	0.04
Older entrant (>21 years):
UKCAT in borderline cases v as factor	0.32 (0.08 to 1.25)	0.1	0.22
UKCAT as threshold v in borderline cases	1.46 (0.34 to 6.22)	0.6
UKCAT as threshold v as factor	0.47 (0.13 to 1.65)	0.2
Attendance at state (non-grammar) school	3.47 (1.89 to 6.35)	<0.001
Academic attainment (standardised tariff score in SDs)	0.13 (0.06 to 0.29)	<0.001
Non-white ethnicity	0.22 (0.04 to 1.25)	0.09
Standardised UKCAT total score (SDs)	0.74 (0.51 to 1.07)	0.1
English as second language	0.60 (0.20 to 1.79)	0.4
Socioeconomic class 4 or 5†	1.34 (0.36 to 5.05)	0.7
Male sex	0.92 (0.54 to 1.57)	0.8
UKCAT in borderline case (v as factor)-academic attainment interaction	3.94 (1.67 to 9.29)	0.002
UKCAT as threshold (v as factor)-academic attainment interaction	2.77 (1.25 to 6.11)	0.01
UKCAT in borderline cases (v as factor)-non-white ethnicity interaction	8.77 (1.12 to 68.71)	0.04
Academic attainment-English as second language	6.25 (2.54 to 15.41)	<0.001
Academic attainment-school type interaction	0.40 (0.22 to 0.75)	0.004

ICC=intraclass correlation; UKCAT=UK clinical aptitude test. Baseline category was group of medical schools using UKCAT score as a factor in admissions process, unless stated otherwise.

*Represents residual correlation of latent responses within each university.

†Office for National Statistics socioeconomic classification.

Table 12 depicts the results of multilevel multiple logistic regression for the prediction of widening participation status of entrants for school type attended, socioeconomic status, and achieving below average academic attainment (for this cohort of medical school applicants). Several trends that were highly statistically significant on estimation of the raw odds ratio were now absent or of only borderline significance. Notably, the intergroup differences in the probability of an entrant being from a state school (as opposed to independent or grammar school) attenuated to a difference of only borderline significance (odds ratio 1.60, 95% confidence interval 0.97 to 2.62, P=0.06) for medical schools using UKCAT as a threshold compared with those using UKCAT in borderline cases. In contrast, the faint trends for entrants to medical schools using UKCAT as a factor and as a threshold to be more likely to be from low socioeconomic backgrounds were now of borderline significance (P=0.05 and P=0.06, respectively). The relatively high proportions of entrants to medical schools using UKCAT as a factor with below average academic attainment compared with the other categories of medical schools remained statistically significant, even after adjustment for institutional and potential confounding effects.

Table 12

 Results of multilevel multiple logistic regression analyses for prediction of school type attended, socioeconomic status, and relatively low academic attainment (less than AAB or equivalent at A level) of entrants to 22 participating medical schools with complete data available (n=2679)

Predictor	Odds ratio (95% CI)	P value	ICC*
State or non-grammar school:
UKCAT in borderline cases v as factor	0.82 (0.51 to 1.30)	0.5	0.04
UKCAT as threshold v in borderline cases	1.60 (0.97 to 2.62)	0.06
UKCAT as threshold v as factor	1.31 (0.84 to 2.03)	0.2
Age >21 years	3.72 (2.08 to 6.66)	<0.001
Male sex	1.67 (1.20 to 2.32)	0.002
Socioeconomic class 4 or 5†	1.87 (1.15 to 3.02)	0.01
Non-white ethnicity	0.73 (0.57 to 0.94)	0.01
Academic attainment (standardised tariff score in SDs)	0.84 (0.73 to 0.98)	0.02
English as second language	1.39 (1.03 to 1.89)	0.03
Standardised UKCAT total score (SDs)	0.94 (0.82 to 1.07)	0.3
UKCAT as factor (v borderline)-male sex interaction	0.44 (0.29 to 0.67)	<0.001
UKCAT as threshold (v borderline)-male sex interaction	0.59 (0.39 to 0.90)	0.01
UKCAT score-non-white ethnicity interaction	0.70 (0.54 to 0.92)	0.01
Socioeconomic class 4 or 5†:
UKCAT in borderline cases v as factor	0.28 (0.08 to 0.97)	0.05	<0.001
UKCAT as threshold v in borderline cases	3.38 (0.94 to 12.12)	0.06
UKCAT as threshold v as factor	0.95 (0.45 to 2.01)	0.9
Standardised UKCAT total score (SDs)	0.84 (0.61 to 1.16)	0.3
Non-white ethnicity	2.85 (1.33 to 6.13)	0.007
Attendance at state (non-grammar) school	1.31 (0.77 to 2.23)	0.3
Age >21 years	1.59 (0.55 to 4.58)	0.4
English as second language	0.61 (0.20 to 1.81)	0.4
Male sex	1.01 (0.64 to 1.59)	0.9
Academic attainment (standardised tariff score in SDs)	0.98 (0.72 to 1.34)	0.9
School type-English as second language interaction	3.61 (1.05 to 12.41)	0.04
UKCAT in borderline cases (v as factor) use-non-white ethnicity interaction	4.66 (1.06 to 20.46)	0.3
Below average academic attainment:
UKCAT in borderline cases v as factor	5.19 (2.02 to 13.33)	0.001	0.17
UKCAT as threshold v in borderline cases	0.72 (0.28 to 1.91)	0.5
UKCAT as threshold v as factor	3.76 (1.50 to 9.38)	0.005
Standardised UKCAT total score (SDs)	0.66 (0.56 to 0.78)	<0.001
Age >21 years	16.01 (4.43 to 57.83)	<0.001
Male sex	1.47 (0.94 to 2.28)	0.09
Socioeconomic class 4 or 5†	1.39 (0.78 to 2.50)	0.3
English as second language	0.99 (0.67 to 1.47)	0.9
Non-white ethnicity	1.02 (0.76 to 1.37)	0.9
Attendance at state (non-grammar) school	1.02 (0.82 to 1.27)	0.9
UKCAT in borderline cases (v as factor)-age >21 years interaction	0.07 (0.02 to 0.27)	<0.001
UKCAT as threshold (v as factor)-age >21 years interaction	0.18 (0.06 to 0.60)	0.005
UKCAT as threshold (v as factor)-male sex interaction	0.51 (0.30 to 0.89)	0.02
School type-age >21 years interaction	4.10 (1.19 to 14.09)	0.03

ICC=intraclass correlation. Baseline category was group of medical schools using UKCAT as a factor in admissions process, unless stated otherwise.

*Represents residual correlation of latent responses within each university.

†Office for National Statistics socioeconomic classification system.

Modelling the prediction of achieving conditions for medical school entry

Of 16 435 total applicants, 1189 were initially given 1507 unconditional offers, having already achieved the required academic conditions at the time of application. A further 5368 candidates were initially provided with 7363 conditional offers (an average of 1.4 offers per successful candidate). Of these individuals, 4470 (83%) eventually obtained an unconditional offer, having obtained sufficient academic qualifications to satisfy the relevant institutions that entry was merited. In most, but not all cases, this would involve achieving the initial A level (or equivalent) grades set as part of the conditions of the provisional offer. To explore the predictors of an application being converted from a conditional to an unconditional offer data were analysed from 3290 applications relating to 2803 candidates holding conditional offers where complete data were available. A multilevel multiple logistic regression model was built up in a stepwise fashion. A random intercept for each candidate was introduced, although it was not possible to model the random effects associated with individual institutions simultaneously. All possible combinations of interactions were explored and those found to be significant and independent predictors of successfully converting a conditional offer to an unconditional one were included in the final model (table 13). The strongest predictor of success was academic attainment (odds ratio 10.35, 95% confidence interval 8.01 to 13.27, P<0.001), in these cases measured using a standardised UCAS tariff. Indeed, the only other significant and independent predictor was the application being associated with a medical school using UKCAT in borderline cases or a factor in the admissions process (compared with the remaining group). However, a trend of borderline statistical significance was observed for conditional offers associated with candidates of low socioeconomic status to have less probability of being converted to unconditional ones (0.42, 0.18 to 0.99, P=0.05). Two interactions were also significant. Firstly, an interaction between mature applicant status and academic achievement was observed (0.11, 0.07 to 0.17, P<0.001). This is interpreted as increased A level achievement being relatively less of an advantage to those applying to holding conditional offers and over 20 years old, compared with younger applicants. In practice many mature candidates will be applying on the basis of a university degree, either achieved or pending, rather than A level achievement, and therefore this finding is readily understood in this context. Perhaps a less anticipated interaction was that observed for low socioeconomic status and non-white ethnicity (8.56, 1.81 to 40.43, P=0.007). This implied that those applicants holding conditional offers from low socioeconomic backgrounds were more likely to convert them to unconditional ones if they were of non-white than of white ethnicity.

Table 13

 Results of multilevel multiple logistic regression analysis for the prediction of achievement of grades or degree required for a conditional offer to be converted to an unconditional offer. Data from 3290 applications relating to 2803 candidates holding conditional offers where complete data were available were analysed

Predictor variable	Odds ratio (95% CI) for receiving offer	P value	ICC*
Academic attainment (standardised tariff score in SDs)	10.35 (8.01 to 13.27)	<0.001	<0.001
Socioeconomic class 4 or 5†	0.42 (0.18 to 0.99)	0.05
Academic attainment-age interaction	0.11 (0.07 to 0.17)	<0.001
Non-white ethnicity-socioeconomic class 4 or 5† interaction	8.56 (1.81 to 40.43)	0.007
Type of medical school group applied to:
UKCAT in borderline cases	1.41 (1.05 to 1.89)	0.02
UKCAT as factor	1.51 (1.10 to 2.08)	0.01
UKCAT as threshold	1	NA
English as second language	1.50 (0.94 to 2.41)	0.09
Male sex	0.84 (0.66 to 1.07)	0.2
Standardised UKCAT total score (SDs)	1.13 (0.96 to 1.35)	0.2
Age >21 years	1.67 (0.76 to 3.63)	0.2
Non-white ethnicity	0.88 (0.64 to 1.21)	0.4
Attendance at state (non-grammar) school	0.92 (0.72 to 1.18)	0.5

ICC=intraclass correlation; NA=not applicable.

*Represents residual correlation of latent responses for each applicant.

†Office for National Statistics classification system.

Missing data analysis

Those applications with no data on socioeconomic background of the candidate (n=12 562) were significantly less likely to result in an offer (0.81, 0.76 to 0.85, P<0.001). Candidates with missing socioeconomic status (n=4605) were significantly more likely (P<00.01) to be male (1.13, 1.05 to 1.21), have attended a private or grammar school (1.13, 1.05 to 1.21), be over 21 years at application (1.33, 1.23 to 1.44), be of non-white ethnicity (1.73, 1.62 to 1.86), speak English as a second language (1.39, 1.28 to 1.51), and have below average academic attainment (1.27, 1.17 to 1.39) and UKCAT score when compared with those with socioeconomic background reported.

Likewise, those applications with no attainment data on A levels, higher, or Irish leaving certificate (n=11 949) were significantly less likely to result in an offer (0.58, 0.54 to 0.61, P<0.001). Candidates with missing or non-standard advanced qualification data (n=4917) were significantly more likely (P<0.01) to have missing data on socioeconomic status (1.26, 0.17 to 1.36), be female (1.10, 1.03 to 1.18), have attended a state school (3.65, 3.36 to 3.96), be over 21 years at application (45.28, 40.61 to 50.49), be of non-white ethnicity (1.42, 1.32 to 1.52), speak English as a second language (2.15, 1.99 to 2.33), and have a below average UKCAT score when compared with those with socioeconomic background reported (1.86, 1.74 to 1.99). Thus, according to missing data theory, the missing socioeconomic status and advanced qualification data seemed to be not missing completely at random—that is, the value of the missing variable and the probability that it is missing is unrelated to observed variables.

The results of the sensitivity analyses using multiple imputed data generally indicated that the missing values for advance qualification as socioeconomic status were missing at random. On average the odds ratio recovered from the analyses of the imputed data only varied by around 5-10% compared with those obtained for the non-imputed data. The only exception to this was the findings for the prediction of conditional offers using imputed advanced qualification values. These results differed by a slightly larger degree, with, on average, odds ratio differing by an average of 12%. A fuller report of these results is available from the lead author’s website (www.dur.ac.uk/p.a.tiffin/fps).

Discussion

The main findings of this study at least partially supported our original hypothesis; the results suggest that important differences exist, both in terms of the probability of an offer and that of finally taking up a place at medical school according to certain sociodemographic characteristics of the candidates and how the medical school applied to used the UKCAT. In terms of offers, the negative effect of being a member of a widening participation group seemed to be mitigated by the admissions processes utilised by medical schools using the UKCAT score as a threshold for interview or offer decisions. Indeed, only academic achievement and UKCAT score were significant and independent predictors of either a conditional or an unconditional offer at this category of medical school (tables 5 and 6). The findings from these separate models were largely supported by the interactions observed in the combined analyses (tables 7 and 8). Medical schools using the UKCAT score as a factor in the admissions process could largely be seen as an intermediate category, with some exceptions. For instance such medical schools were less likely than those that used UKCAT as a threshold to provide a conditional offer to a candidate from a state school or a low socioeconomic background (table 8) but were also more likely to provide a conditional offer to an individual with below average academic achievement (table 7). Heavier use of UKCAT therefore seemed to promote widening access as intended, at least to a degree, in terms of offers.

Although it is important that universities act as fairly as possible when providing offers, ultimately it is the characteristics of the candidates finally taking up places at universities that will determine the demographics of the UK medical workforce. When analysing the raw proportions of under-represented groups it appeared that many of the trends observed in the offer data were translated into final uptake of placement. However, after controlling for individual institutional effects and the potential confounding of other sociodemographic variables the picture shifted somewhat. After these adjustments the clearest remaining differences were that entrants to medical schools that used the UKCAT weakly (in borderline cases) were less likely to be male than for the other two categories of medical school but were more likely to have English as a second language than those taking up places in medical schools that used UKCAT as a factor (table 11). Several potentially important trends, albeit of borderline statistical significance, were also observed in that those medical school using the UKCAT as a threshold score were more likely to admit students from state or non-grammar schools and from low socioeconomic backgrounds than were medical schools that used UKCAT in borderline cases (table 12). This latter trend for entrants of low socioeconomic status was also observed for medical schools using UKCAT as a factor compared with those using it in borderline cases. A trend of borderline statistical significance was also observed for applicants from low socioeconomic backgrounds holding conditional offers to be less likely to achieve the grades necessary for entry, although this effect might apply less to those of non-white ethnicity. This in turn might weaken the impact of UKCAT use on the final proportion of entrants from low socioeconomic backgrounds.

Comparison with previous findings and possible interpretations

After controlling for factors such as UKCAT and academic performance we estimated that non-white applicants were still around 40% less likely to receive a conditional offer from a medical school using UKCAT in borderline cases than a candidate reporting their ethnicity as white (table 5); an almost identical finding to that reported by a study of ethnic minority applicants to medical school in 1990.27 A subsequent study from candidates for admission from 1996 and 1997 also reported a relative disadvantage for applicants from ethnic minority groups, although the authors noted variation between individual medical schools unrelated to the overall number of non-white applicants to each institution. The present results from the separate models were largely echoed by the pooled analysis, which directly compared the three types of medical school. Some authors have cited such findings as evidence of institutional discrimination.28 Although UCAS does not pass on information of ethnicity to universities it may be apparent from a candidate’s name or subsequently at interview. Evidence of racial discrimination has been uncovered previously within a medical school setting and as such cannot be entirely ruled out, deserving further exploration.29 It has been previously highlighted that racism is defined as an attitude, and thus studies, such as the present one, that simply examine the probabilities of an offer to an applicant are unable to provide direct evidence of this.30 Moreover, there may be more innocent, unmodelled effects that may explain some of these observations. For example, it may be that candidates from certain under-represented groups are less likely to have a relative in the medical profession and therefore find it more difficult to gain relevant work experience that would bolster their application and improve their performance at interview. One possibility is that universities using the UKCAT as a threshold may place less emphasis on personal statements or the interview in deciding on offers and mitigating against this effect during the offer process (tables 7 and 8). Owing to the nature of the structure of offer data, individual institutional effects could not be controlled for at the same time as within candidate effects. However, whether these observations are associated with a group effect or individual universities they deserve further exploration.

The relative over-representation of women entering medical school is a sensitive and contentious issue. The proportion of women entering medical school in the United Kingdom and North America has steadily risen; in the United Kingdom most general practitioners are forecast to be women by 2013 and most of all doctors by 2017.31 Some have argued against this disparity,32 highlighting that the current situation may increase the risk of future workforce planning difficulties as women tend to take up posts in specialties perceived as more family friendly, such as general practice,33 and are more likely to work part time.34 In contrast, it has been suggested that female doctors are more likely to exhibit an empathic and caring communication style, valued by patients, and that women perform better than men in both undergraduate and some postgraduate examinations.35 In the present study, participating universities using the UKCAT as a threshold score had an average intake of 50% male entrants, which is representative of the general population (table 9). Thus, those in favour of reducing the excess of female medical students may choose to support UKCAT used in this manner in the admissions process.

Candidates with relatively low UKCAT scores may have a greater tendency to apply to universities with weak use of the test (table 2). However, the absolute differences in the average test scores between candidates associated with applications to the three types of medical school were small. This implies that, at least in 2009, few applicants were making their university choices on the basis of UKCAT policy. It is important that they have the opportunity to do so, as we have shown it can have an important impact on who receives offers.

More non-native English speakers were in medical schools with the weakest use of the UKCAT (tables 9 and 11). Performance on the UKCAT was poorer in those with English as a second language, and therefore a more robust use of the test in the admissions process could potentially disadvantage such candidates; although it is worth noting that once ethnicity was controlled for no such trend was noted for offers (tables 5-8). In addition, entrants to medical schools using UKCAT as a factor were less likely to have below average academic achievement compared with their peers.

In terms of sex, generally males perform better than females on the UKCAT16 and this may mitigate against their long established disadvantage when applying for a place at medical school. However, this situation is more complex, in that women who do well at the UKCAT gain a stronger advantage than males. Interestingly, the relative excess of males observed entering medical schools that had a stronger use of the UKCAT remained even after controlling for individual institutional effects and UKCAT total score. This suggests that it was the way the test scores were being used, rather than absolute sex differences, that underlay this finding.

Universities with the weakest use of the UKCAT (in borderline cases) were most likely to admit entrants with relatively low school attainment. This observation is not easily interpreted, although several possible explanations exist. Firstly, these medical schools sometimes used the UKCAT as a rescue mechanism, interviewing or making offers to candidates who would not normally be considered, for example, owing to low predicted academic attainment. Therefore the use of the UKCAT in this way may compensate for relatively low A level performance in some cases. However, data held by the UKCAT consortium suggest that, in practice, this mechanism is invoked relatively rarely and therefore is unlikely to fully account for this observation. A second explanation may be that the intention to use the UKCAT in a rescue manner may be a marker of an institutional attitude towards not prioritising academic achievement over other candidate attributes. Thus, in this case the manner in which UKCAT was used would be acting as an instrumental variable. Such institutions may be more likely to set lower academic conditions for entry or show latitude to candidates who eventually did not quite achieve these required grades. Lastly, it may be pure coincidence that, on average, the medical schools that used the UKCAT in this manner happened to have had slightly lower entry requirements compared with other medical schools in 2009.

Several mediators could potentially explain the disparities observed between data relating to offers and those concerning medical school entrants. Firstly, applicants holding more than one unconditional offer could choose which, if any, of the universities to enter, and this is not something that medical school can influence. Secondly, a minority of applicants holding conditional offers may fail to achieve the examination results required for admission. Indeed, those from a low socioeconomic background may be over-represented in this category (table 13). Thirdly, it was not possible to control for the individual (random) effects associated with specific medical schools applied to while adjusting for the dependency of observations nested within each candidate. Therefore it is possible that some effects that appear associated with how UKCAT is used were, at least, partly attributable to the behaviour of individual universities. For example, to draw a comparison with the entrant data, not all medical schools using UKCAT as a threshold admitted a higher than average proportion of students from low socioeconomic backgrounds (fig 2). Such variability was also present in the offer data to some extent (see standard deviations, table 1). These differences are likely to be due to variations in admissions policies aside from UKCAT usage. The effects would have been unmasked once an adjustment for university level effects was possible in the data on entrants.

Strengths and limitations of the study

This is the first study to report in detail on the impact of the UKCAT on medical school selection. Missing data were sparse except for two predictor variables: socioeconomic background and school qualifications. Therefore some caution must be exercised when drawing conclusions about the role of these predictors. However, the findings from the sensitivity analysis using multiple imputation suggested that the data were largely missing at random and thus should not affect the conclusions drawn from the multilevel multiple logistic regression models predicting offers. The case of missing advanced qualification data in relation to the prediction of conditional offers could be considered something of an exception to this; when these missing data were imputed the findings showed a relatively greater degree of variation compared with the results for the non-imputed data. Although graduate and other widening access courses make up a small fraction of overall UK medical school intake, these findings are unlikely to generalise to this population of applicants, who were excluded from the analysis.

Concerning the analysis of offers; it was not possible to control for the random effects associated with specific universities applied to while also adjusting for the dependency of observations nested within each candidate. This was because of the offer data structure being hierarchical but not nested—that is, more than one candidate could apply to more than one university. These effects would be particularly pronounced if, for example, a particular medical school had a strong policy of recruiting students from one or more widening participation categories. However, the intraclass correlation values observed in the entrant data, where university effects could be explored, were relatively low (range 0.22 to <0.001) suggesting that most, if not all, of these within medical school effects were relatively trivial once the association with education and demographic variables were controlled for, although caution must be exercised in extending this assumption from the entrant to the offer data.

One of the most striking findings was how small the absolute numbers of applicants and entrants to medical school were from low socioeconomic backgrounds; only around 5.5% of applicants and 4.5% of entrants reported being from socioeconomic class 4 or lower socioeconomic background (although this may be mildly underestimated owing to missing data).25 Because of these small numbers the study may have lacked power to detect a statistically significant difference between categories of medical school, although trends of borderline significance between those using UKCAT in borderline cases and those with stronger usage were detected.

It could be argued that the use of the UKCAT may be an “instrumental variable” in that it could represent a marker of a pro-widening participation attitude of a medical school, rather than be the main cause of an increase in candidates from a widening participation group being offered places. Although this cannot be absolutely disproved, the UKCAT specific effects observed in the models would tend to counter this hypothesis. Moreover, if several institutions have changed their use of the test over the past few years it would be important to explore whether the level of disadvantage encountered by widening participation groups also altered in line with this. This would be compelling evidence against the case for the mode of UKCAT use as an instrumental variable. Moreover, the effect of the predominant use of the UKCAT by each group or individual institutions might mask more subtle effects such as the use of the UKCAT as a rescue mechanism for certain subgroups of candidates or the effect of the level of the threshold score selected.

Lastly, it has been pointed out that statistical models are useful precisely because they omit certain variables.36 As such they provide simplifications of reality. Admissions processes are complex and unique and for this reason the models may have limited predictive power on an individual level. However, the absolute proportions of each type of applicant from a widening participation group observed to be taking up a place at medical school would support the models developed as being generally valid for this cohort studied.

Conclusions and implications for policy

Is the creation of a population of medical students that is demographically more representative of the United Kingdom as a whole likely to have entirely positive implications for the delivery of British healthcare? Some would argue that medical schoold should focus on admitting those most likely to make the “best” doctors, and therefore medical student populations may not necessarily be representative of the populations from which they are drawn. To date, research has generally defined promising medical school entrants in negative terms—that is, those who are least likely to fail undergraduate or postgraduate examinations or face future professional disciplinary action. In this respect, controversially, one could argue for the status quo (an excess of women, those with good academic performance, and those of a higher socioeconomic status), as male sex, lower social class, and poor academic performance have been reported to be associated with subsequent professional disciplinary action.37 38 However, research from the United States suggests that students drawn from minority populations enrich the teaching environment of a medical school39 and may be more likely to practice in areas underserved by healthcare facilities.40 In addition, patient satisfaction with their doctor may be increased in cases where they share the same ethnicity41 or sex42 and, although research is lacking, this finding could potentially extend to other sociodemographic characteristics. Also, the issue of social equity and fair opportunities for all applicants of comparable ability cannot be overlooked, and the present findings suggest that the robust use of the UKCAT is a potential tool for dealing with such disadvantage.

In summary, our findings suggest that placing an increased weight on an applicant’s UKCAT performance significantly reduces the disadvantage faced by most candidates from under-represented groups when seeking an offer of a place from a medical school. These benefits do not always seem to translate into proportions of entrants from ethnic minority groups; candidate choice and academic performance are likely to be important mediating factors between offer and acceptance and these are factors over which universities will have minimum influence. Nevertheless, medical schools have a duty to act in as fair a way as possible during the admissions process, and the UKCAT seems to be a tool that has the potential to produce a more level playing field for many of those seeking access to the medical profession.

The introduction of the UK clinical aptitude test (UKCAT) was part of an initiative to broaden the demography of medical school entrants

UKCAT scores are modestly correlated with A level attainment, and performance is associated with white ethnicity, professional or managerial background, independent or grammar school education, and male sex

It is unknown whether the UKCAT has tackled the relative disadvantage of certain socioeconomic groups when applying to UK medical schools and how this impact might vary according to the use of the test in the admissions process

Being from an under-represented socioeconomic group was not a major disadvantage for candidates applying to medical schools that used UKCAT as a threshold, after controlling for academic attainment and UKCAT score; this was not so for applications to universities using the test more weakly

Stronger use of the test (threshold score) was associated with increased proportions of males, possibly state school students, and those from low socioeconomic status backgrounds entering medical school

Weaker use of the test (borderline cases, weighted factor) was associated with higher proportions of entrants with relatively low academic attainment and English as a second language