Which factors predict candidate outcome in advanced life support courses? A preliminary observational study.

PURPOSE: To identify factors associated with candidate outcome in the European Resuscitation Council (ERC) advanced life support (ALS) provider courses.
METHODS: Medical doctors participating as candidates to consecutive ALS courses organised by an ERC training centre in Italy were enrolled in this prospective cohort study. The association between the ALS course outcome and candidate demographics, professional background and pre-course knowledge measured by using the pre-course multiple choice quiz (MCQ) was investigated by using logistic regression.
RESULTS: A total of 283 candidates, median age 31 years, were evaluated. Among them, 269 (95.1%) passed the final evaluation and 14 (4.9%) failed. Candidates who passed were younger (median age 31 vs. 37.5 years; p = 0.006) and attained a higher pre-course MCQ score (median 84 vs. 72.5%; p < 0.0001). On multivariate analysis, a higher pre-course MCQ score (OR 1.18 [95%CI 1.09-1.28]) and a basic life support (BLS) certification (OR 5.00 [95%CI 1.12-22.42]) were independent predictors of candidate success, while older age was associated with a significantly higher risk of failing (OR 0.90 [95%CI 0.83-0.97]). Female candidates had higher pass rates (97.2 vs. 91.2%; p = 0.048); however, after correction for confounders gender was not significantly associated with candidate outcome. Neither candidate specialty nor site of work was a predictor of candidate success.
CONCLUSIONS: On ALS courses, younger age and a higher level of specific pre-course knowledge, as measured by both the pre-course MCQ and the presence of BLS certification, are the most important predictors of success. Candidate gender and professional background did not show a significant correlation with course outcome.

Introduction

In most European countries, the European Resuscitation Council (ERC) advanced life support (ALS) provider course is recognised as the gold standard to prepare members of multidisciplinary teams to manage both cardiac arrest and peri-arrest situations [1, 2]. Resuscitation and initial management of the acutely ill patient are among the twelve domains of the core competencies required of a specialist in intensive care medicine [3].

Unfortunately, some ALS candidates fail to achieve their ALS certification. Failure rates in the cardiac arrest scenario test (CASTest) at the end of the ALS course may be as high as 60% [4]. Until now, no specific investigation has been conducted to identify factors associated with ALS course outcome. Knowing in advance those factors would be useful for two reasons:

At the beginning of an ALS course, it would allow the course faculty to identify early on those candidates who are at risk of failure and who need additional support;

As a general policy, it would help in targeting the ALS course to the most appropriate candidates, i.e. those professional figures who are most likely to benefit from ALS training.

The aim of this study was to investigate whether pre-course factors, such as candidate age and gender, professional background and pre-course knowledge, can predict the outcome of the ALS course.

Materials and methods

This was a prospective, single-centre cohort study.

Candidates and data collection

All medical doctors participating in the ALS provider courses organised by the Catholic University School of Medicine (UCSC) Training Centre in Rome between 1 November 2006 and 30 June 2009 who did not have a previous ALS certification were included in the study. The UCSC Training Centre operates within the ALS national training network of the Italian Resuscitation Council (IRC), an ERC approved school.

Candidates gave their consent for anonymous use of both their demographic information and course results for scientific investigational purposes on applying for the ALS course via the UCSC course management website (www.corsi-rianimazione.it). The study was approved by the ethics committee of the UCSC (protocol number A598).

The ERC ALS provider course

The ALS provider course has been described elsewhere [1]. It is a residential teaching and training course designed for healthcare professionals and focused on prevention of cardiac arrest, management of cardiac arrest and peri-arrest situations and stabilisation of the resuscitated patient until transfer to a critical care area.

Before the ALS course, a 120-item multiple choice quiz (MCQ) is usually sent to the candidates along with the manual to allow self-assessment of candidates’ pre-course knowledge. For the purpose of this study—in order both to standardise testing and to avoid reference to the manual—the pre-course MCQ was given on candidates’ arrival at the training centre and invigilated. A maximum of 60 min was allowed to complete the test.

During the first day of the course, candidates are assessed on airway management and initial assessment and resuscitation (IAR) and must demonstrate competence on both these skills before the final assessment in order to achieve their ALS certification.

Final assessment of the ERC ALS course includes both a 120-item final MCQ and a cardiac arrest simulation test (CASTest). The final MCQ must be completed in 60 min and it has a pass mark of 75%. During CASTest, candidates are assessed on their ability to manage a simulated cardiac arrest scenario as team leaders. Candidates who fail either the final MCQ or the CASTest are entitled to a second attempt (retest) and, if still unsuccessful, are failed. Those who are successful in both the final MCQ and the CASTest (either at first or at second attempt) are certified as ALS providers.

Statistical analysis

The following variables were considered as possible predictors of the course outcome: age; gender; specialty; site of work; previous certification in basic life support (BLS) courses for healthcare providers; and pre-course MCQ score. Univariate analysis was carried out to investigate predictors of course outcomes. Medical specialties that were more pertinent to the subject of the ALS course, such as anaesthesiology, cardiology and emergency medicine, were compared with all the other ones. Chi square or Fisher’s exact test and Mann–Whitney tests were applied for categorical and continuous data, respectively. Statistical significance was set at p < 0.05.

Stepwise multivariate logistic regression was performed by entering into the model all the variables whose p value was less than 0.25 in the univariate analysis. The model goodness of fit was assessed with the Hosmer–Lemeshow test. Statistical analysis was carried out using SPSS, release 12.0 for Windows.

Results

A total of 290 medical doctors participating in 13 consecutive ALS courses were considered for inclusion in the study. Four of them were excluded due to a previous ALS certification. Three candidates dropped out of the ALS course before completion for personal reasons and were also excluded.

The remaining 283 candidates (median age 31 years; 64.0% females) were included in the study. All candidates in our population demonstrated sufficient competence on both airway management and IAR by the end of the first day. As regards final evaluation, 269 (95.1%) candidates passed and 14 (4.9%) failed. Ten of these 14 candidates attained a sufficient score on the final MCQ but failed the CASTest, while four failed both the CASTest and the MCQ. Candidates’ characteristics are summarised in Table 1.

Table 1

Characteristics of the study subjects

Number of subjects	283
Median age, years (range)	31 (25–57)
Females	181 (64.0)
Specialist	173 (61.2)
Resident	110 (38.8)
Specialty
Anaesthesiology	77 (27.2)
Internal medicine	41 (14.5)
Cardiology	32 (11.3)
Infectious diseases	20 (7.1)
Surgery	12 (4.2)
Respiratory medicine	12 (4.2)
Emergency medicine	10 (3.5)
Geriatric medicine	9 (3.2)
Other	70 (24.7)
BLS certification
BLS provider	134 (47.9)
BLS instructor	13 (4.6)
Site of work
Ward	128 (45.2)
Operating room	58 (20.2)
ICU	29 (10.6)
Clinic	38 (13.4)
ED	21 (7.4)
Ambulance	9 (3.2)
Course outcome
Passed	269 (95.1)
Failed	14 (4.9)

Values are numbers (percentages) unless otherwise stated

ICU intensive care unit, ED emergency department

In comparison with those who failed, candidates who passed were significantly younger (31 vs. 37.5 years; p = 0.006; Table 2), and attained a significantly higher median score at the pre-course MCQ (84 vs. 72.5%; p < 0.0001). On multivariate analysis (Table 3), a higher pre-course MCQ score and a BLS certification were associated with significantly higher probabilities of passing the final test (OR 1.18 [95%CI 1.09–1.28] and OR 5.00 [95%CI 1.12–22.42] respectively), while older candidates had significantly lower probabilities of passing (OR 0.90 [95%CI 0.83–0.97]). Women had a significantly higher pass rate (97.2 vs. 91.2%; p = 0.048) but on multivariate analysis gender was not an independent predictor of outcome.

Table 2

Results of univariate analysis comparing passed and failed candidates

	Passed (n = 269)	Failed (n = 14)	p Value
Median age, year (range)	31 (25–57)	37.5 (27–54)	0.006
Gender
Male	93 (91.2)	9 (8.8)	0.048
Female	176 (97.2)	5 (2.8)
Medical specialties
Anaesthesiology, cardiology or emergency medicine	114 (95.8)	5 (4.2)	0.78
All other specialties	155 (94.5)	9 (5.5)
Site of work
Ward/clinic	155 (93.4)	11 (6.6)	0.27
Operating room	57 (98.3)	1 (1.7)
ICU/ED/ambulance	57 (96.6)	2 (3.4)
BLS certification
Yes	147 (97.4)	4 (2.6)	0.1
No	122 (92.4)	10 (7.6)
Median score at the pre-course MCQ, % (range)	84 (54–100)	72.5 (37–89)	<0.0001

Values are numbers (percentages) unless otherwise stated

Table 3

Predictors of course pass according to multivariate logistic regression

Variable	Odds ratio (95% CI)	p Value
Pre-course MCQ score (%)	1.18 (1.09–1.28)	0.0001
BLS certification	5.00 (1.12–22.42)	0.03
Age (years)	0.90 (0.83–0.97)	0.008

Hosmer–Lemeshow goodness-of-fit = 0.456

Discussion

Candidate age, BLS certification and pre-course knowledge, as measured by the pre-course MCQ, were the most important predictors of ALS course outcome in our population.

Increasing age was associated with a lower probability of passing the final evaluation. Probably, in older candidates a higher decay of their knowledge about emergency medicine may have occurred, due to the longer time passed between their degree and the ALS course. Another cause could have been an age-related decay in learning capability. It has been estimated that learning capability declines by a 1% per year in adults from 22 to 50 years of age [5]. Age-related losses are substantial in ‘working memory’ tasks, which usually require the maintenance of some information while dealing concurrently with further incoming stimuli—as occurs during an ALS scenario [6, 7]. Furthermore, it is also possible that older candidates may have not been familiar or comfortable with the scenario-based/role play-type teaching as it is a relatively new methodology.

High scores on the pre-course MCQ and a BLS certification were independent predictors of course pass in our study. We think that previous BLS training may have influenced the ALS course outcome in two ways: first by teaching candidates a correct approach to cardiovascular emergencies; secondly because it may have increased candidates’ confidence with scenario teaching and assessment. Although BLS certification is currently not required for access to ERC ALS courses, results of our study suggest that proficiency in BLS should be recommended to ALS course participants.

We administered the pre-course MCQ on arrival in standardised and invigilated conditions, which is not routine practice of the ALS course. This, however, was necessary to ensure that pre-course MCQ results would reliably reflect the candidates’ pre-course knowledge. Results of our study, if confirmed, would suggest that the pre-course MCQ may be used at the beginning of the course to identify early on those candidates at risk of failure who may benefit from additional support during the course.

Pre-course knowledge about cardiac arrest may also be a result of candidate professional background. One can expect higher ALS course pass rates in those professional groups more likely to be involved in management of cardiac arrests, such as anaesthesiologists, cardiologists, emergency physicians and those working in ICUs. However, neither candidate specialty nor site of work showed a significant association with candidate success or failure in our population, which suggests that professional background is less important for ALS course success than specific pre-course preparation as measured by the pre-course MCQ. Our results are in accordance with those of Napier et al. [8] who documented no differences in CASTest pass rates between professions in UK ALS course candidates.

The final pass rate in our study (95.1%) is similar to the national average (97%) reported for ALS courses in Italy [9]. We are not aware of national reports on ALS outcome in other countries. In a large group of UK course centres, Perkins et al. [4] reported a pass rate ranging from 40 to 93%. However, these data refer only to the outcome of the CASTest pass at first attempt, therefore not including retests. The international course management system (http://courses.erc.edu/) recently introduced by the ERC will certainly represent an important research tool to monitor and compare the outcome of ALS courses internationally.

Study limitations

Our study has a series of limitations. Firstly, its single-centre nature may limit its generalizability. We cannot exclude that results of this study may have been affected by the specific features of our group of candidates and instructors. A recent study [4] documented how the outcome of ALS courses can vary—at least in terms of pass rates—among training centres. Larger, multicentre studies are needed to confirm our findings.

Secondly, this study was not blinded. In fact, all the predictive factors we analysed, including the results of the pre-course MCQ, were known by the course faculty and this may in theory have biased the course results. However, concealing candidate data from the faculty would have been impractical, since those data are needed by the ALS course faculty in order to provide feedback and support to candidates during the course. To overcome this limitation, future studies should allocate the CASTest evaluation to independent assessors.

Thirdly, our study was limited only to medical candidates, therefore excluding nurses. This is because in Italy very few nurses apply to ALS provider courses. Future multicentre studies should address whether specific predictive factors exist for the nurse population.

Conclusions

Results of our study showed that younger age, a previous BLS certification and a higher pre-course knowledge, as measured by the pre-course MCQ administered on arrival, were independent predictors of success for medical doctors participating in ALS courses. Candidate professional background did not show a significant association with ALS course outcome. Larger, multicentre studies are needed to confirm the results of this preliminary investigation.