Procedural aspects of the organization of the comprehensive European Board of Ophthalmology Diploma examination.

The comprehensive European Board of Ophthalmology Diploma (EBOD) examination is one of 38 European medical specialty examinations. This review aims at disclosing the specific procedures and content of the EBOD examination. It is a descriptive study summarizing the present organization of the EBOD examination. It is the 3rd largest European postgraduate medical assessment after anaesthesiology and cardiology. The master language is English for the Part 1 written test (knowledge test with 52 modified type X multiple-choice questions) (in the past the written test was also available in French and German). Ophthalmology training of minimum 4 years in a full or associated European Union of Medical Specialists (UEMS) member state is a prerequisite. Problem-solving skills are tested in the Part 2 oral assessment, which is a viva of 4 subjects conducted in English with support for native language whenever feasible. The comprehensive EBOD examination is one of the leading examinations organized by UEMS European Boards or Specialist Sections from the point of number of examinees, item banking, and item contents.

Introduction

The comprehensive European Board of Ophthalmology Diploma (EBOD) examination has a long tradition and is one of the oldest European medical specialist examinations. The comprehensive EBOD examination has been officially used since 1995 by an increasing number of European countries to assess the level of ophthalmic education of their residents at the end of their training. The purpose of the comprehensive EBOD examination is to ensure a minimal standard of knowledge. It aims at demystifying all procedural aspects of the organization of the comprehensive EBOD examination by providing complete transparency both to future participating candidates as well as to the responsible leaders in ophthalmic education within the various countries of the European Union of Medical Specialists (UEMS) to create a platform for further official recognition of the comprehensive EBOD examination within these countries.

Context of the comprehensive European Board of Ophthalmology Diploma examination

Founded in 1958, the UEMS is the oldest medical organization in Europe and has become the representative organization of the national associations of medical specialists in the European Union (EU) and its associated countries. The 39 specialists sections, among which the UEMS Section of Ophthalmology, are covered within UEMS. The European Boards associated with these UEMS Specialists Sections are dealing with putting standards for training and education within their medical specialty. As such, the European Board of Ophthalmology (EBO) is dealing with harmonization of training standards in ophthalmology in UEMS (associated) countries.

As the level of training in ophthalmology in UEMS (associated) countries is difficult to harmonize and even more difficult to grade, assessment of education level was started by organizing a comprehensive EBOD examination on a yearly basis since 1995 [1,2]. As many European Boards within UEMS have been organizing comprehensive European postgraduate medical assessments [3,4], a dedicated UEMS-Council for European Medical Specialty Assessments (UEMS-CESMA) was created following the tenets of the Glasgow Declaration on February 9, 2007 within the structures of UEMS available from: https://www.uems.eu/__data/assets/pdf_file/0009/1242/Glasgow_Declaration_-_February_2007.pdf. As a result, neurosurgery, nuclear medicine, orthopedics and traumatology, pediatric surgery, pathology, plastic, reconstructive and aesthetic surgery, pneumology, urology, and vascular surgery applied to adhere to UEMS-CESMA.

In 2014, 38 European Boards and Specialist Sections are represented within UEMS-CESMA, although not all of them are organizing European postgraduate medical assessments. The specialists section of anesthesiology has the longest tradition in organizing comprehensive European postgraduate medical assessments since 1984 [2,3,5].

EBO has been represented within UEMS-CESMA since November 2009, and the comprehensive EBOD examination was in 2016, the 3rd largest European postgraduate medical assessment with 619 candidates, after anaesthesiology with over 2,000 candidates, and cardiology with approximately 1,000 candidates [3,4]. However, when considering the initiative to promote in peer-reviewed and book publications, the EBOD examination is the leading specialty [2,6,7, 8, 9]. Until present, only the specialists sections of anesthesiology [5] and hand surgery [10] have published peer-reviewed articles about their European postgraduate medical assessments [3,4].

Comprehensive European Board of Ophthalmology Diploma examination and Fellow of the European Board of Ophthalmology title

The comprehensive EBOD examination is organized under the responsibility of the Education Committee of the EBO. Participation at the comprehensive EBOD examination is, unlike the majority of European postgraduate medical assessments [3,4], only open for candidates of UEMS (associated) countries that fulfill the eligibility criteria. Participation at the comprehensive EBOD examination in most UEMS (associated) countries is on voluntary basis. However, some countries do recognize the comprehensive EBOD examination as being equivalent to or being a replacement of their local national assessment. In Switzerland, the comprehensive EBOD examination was made mandatory.

Ophthalmologists who have acquired the EBO Diploma, which is awarded to successful EBOD candidates once they are officially recognized as specialist in ophthalmology within their UEMS (associated) country, are encouraged to add the title Fellow of the European Board of Ophthalmology after their name.

Eligibility criteria for the comprehensive European Board of Ophthalmology Diploma examination

Anyone who is a certified specialist in ophthalmology in a UEMS (associated) country is eligible to sit the comprehensive EBOD examination, on condition that eligibility is confirmed during the application process by the relevant EBO national delegate from the UEMS (associated) country of the applicant’s training. Since 2004, it was also possible to sit the comprehensive EBOD examination during the last year of residency (or after 4 years of training in the United Kingdom), or after additional formal training in a UEMS (associated) country in the case that the title of specialist was obtained in a non-UEMS (associated) country, and upon approval by the head of Department and the relevant EBO national delegate from the UEMS (associated) country of the applicant’s training (Table 1). All candidates that are eligible according to the above-mentioned eligibility criteria, are accepted to sit the comprehensive EBOD examination provided their application form has been received prior to the registration deadline, and provided the applicable registration fee has been paid by the candidate.

Table 1.

Essential criteria maintained for examiners at the comprehensive European Board of Ophthalmology Diploma examination

	Essential criteria
Language	Fluent English and preferably one or two other European languages
Ophthalmology training	Ophthalmology training of minimum 4 years in a full or associated European Union of Medical Specialists member state
Specialist title	A specialist title in ophthalmology is required
Practice experience	Minimum 5 years practice after obtaining the specialist title is needed
Teaching experience	Experience in residence teaching and training must include one of the following: full or partial activity in residence teaching and training in an academic position OR permanent position in a postgraduate teaching hospital
Knowledge	Expertise to assess in at least one of the main topics of the oral assessment (Part 2)
Assessment experience	Examiners participating for the first time at the comprehensive European Board of Ophthalmology Diploma examination may have the position of an observer

Structure of the comprehensive European Board of Ophthalmology Diploma examination

The comprehensive EBOD examination consists of two parts that are both accessible for all eligible candidates [1], which is a unique and attractive feature of this comprehensive European postgraduate medical assessment. This contrasts with many other comprehensive European postgraduate assessments where Part 2, if organized, is only accessible for candidates that were successful at Part 1 [3,4].

Whereas Part 1 of the comprehensive EBOD examination will focus predominantly on the assessment of knowledge, Part 2 will focus more on assessment of higher-order reflection and professional attitude of candidates. Topics and their item numbers covered in Part 1 of the comprehensive EBOD examination was as follows: optics, refraction, and contact lenses (between 4 and 7 multiple-choice questions [MCQs]); pediatric ophthalmology and strabismus (between 4 and 7 MCQs); external, corneal, and adnexal disease (between 4 and 7 MCQs); glaucoma, cataract, and refractive surgery (between 4 and 7 MCQs); retina, vitreous, and uvea (between 4 and 7 MCQs); neuro-ophthalmology (between 4 and 7 MCQs); orbital disease and oculoplastic surgery (between 4 and 7 MCQs); general medicine relevant to ophthalmology (between 4 and 7 MCQs); ophthalmic pathology, microbiology, and immunology (between 4 and 7 MCQs); and pharmacology and therapeutics (between 4 and 7 MCQs).

The topics and their assessment duration covered in Part 2 of the comprehensive EBOD examination are as follows: optics, refraction, strabismus, pediatric ophthalmology, and neuro-ophthalmology (15 minutes viva discussion, split between 2 examiners); cornea, external diseases, orbit, and ocular adnexa (15 minutes viva discussion split between 2 examiners); glaucoma, cataract, and refractive surgery (15 minutes viva discussion split between 2 examiners); and posterior segment, ocular inflammation, and uveitis (15 minutes viva discussion split between 2 examiners).

The subjects are covered in balanced qualitative (difficulty) and quantitative (number of items) modality. The languages provided for Part 1 of the comprehensive EBOD examination are English (master language, continued in the future), French (translation, will be abandoned in the future) and German (translation, will be abandoned in the future), whereas Part 2 of the comprehensive EBOD examination is conducted primarily in English with support for one other native language of the candidates whenever requested by the candidate and whenever feasible.

Part 1 of the comprehensive European Board of Ophthalmology Diploma examination

Part 1 of the comprehensive EBOD examination is a written paper and pencil test composed of 52 MCQs in a modified multiple independent true-false format (type X) [11] including an active “don’t know” option [2,6-9]. The reason for modification of the type X format relies in the use of negative marking for incorrect answers. Although negative marking for incorrect answers has been abandoned by some European postgraduate medical assessments [5], it still remains in use for other European postgraduate medical assessments [10]. Negative marking for incorrect answers indeed has positive effects on the statistical performance parameters of an assessment. The negative marking for incorrect answers has been introduced in 2010 as a method to continuously improve the quality of assessment. In order not to put too much pressure on the candidates sitting the comprehensive EBOD examination, the Examination Committee has decided to consider the “don’t know” option as a valid choice for candidates and decided not to penalize the candidate for this choice. A score of 0 will be obtained for “don’t know” answers, while a negative mark of -0.5 will be given to incorrect answers [2,8]. However, in type X MCQs with negative marking often a negative mark of -1 is given to wrong answers [10]. It can be expected that such a marking attitude would dramatically influence the confidence level of candidates, reason for which the above-mentioned marking attitude has been adopted for the comprehensive EBOD examination. The scoring rules for each of the true/false items of the modified type X MCQs in Part 1 of the comprehensive EBOD examination are as follows: score of +1 awarded in case only correct answer has been indicated; score of 0 awarded in case only the don’t know option has been indicated; and score of -0.5 awarded in all other cases. Candidates are offered the possibility to choose for the “don’t know” option in case they feel uncertain about the answer to the statement. However, the Examination Committee clearly underlines that although no negative mark will be attributed in such a case, no points will be gained either.

The 52 modified type X MCQs included in Part 1 of the comprehensive EBOD examination will result in 260 answers. Therefore, the theoretical maximal obtainable score for Part 1 will be 260. Although the theoretical minimal obtainable score for Part 1 of the comprehensive EBOD examination would be -130, the Examination Committee has decided that candidates sitting the comprehensive EBOD examination cannot obtain a negative total score for Part 1. After calculation of the total scores for Part 1 for all candidates, the average total score and the according standard deviation is to be calculated. Total scores for Part 1 of all candidates was converted in the past into a 1-10 scale based on a pre-defined conversion table (Table 2). As from the 2017 edition on, scores will be converted into a linear 4-10 scale score between the minimum observed test score (equal to 4) and the maximum observed test score (equal to 10). The pass mark for Part 1 of the comprehensive EBOD examination (pass mark equal to 6) was fixed to one standard deviation below the average total score, and will remain at 6 in the linear conversion scale.

Table 2.

Conversion table of the candidate’s total scores for Part 1 of the comprehensive European Board of Ophthalmology Diploma examination based on the average total score and the SD

Converted score	Range of the average total score	Result of test
1	Less than average -2.5 • SD	Fail
2	Equal to or greater than average -2.5 • SD to less than average -2.25 • SD	Fail
3	Equal to or greater than average -2.25 • SD to less than average -2.20 • SD	Fail
4	Equal to or greater than average -2.0 • SD to less than average -1.5 • SD	Fail
5	Equal to or greater than average -1.5 • SD to less than average -1.0 • SD	Fail
6	Equal to or greater than average -1.0 • SD to less than average -0.50 • SD	Pass
7	Equal to or greater than average -0.5 • SD to less than average	Pass
8	Average to less than average +0.50 • SD	Pass
9	Equal to or greater than average +0.50 • SD to less than average +1.0 • SD	Pass
10	More than average +1.0 • SD	Pass

Average, average total score; SD, standard deviation.

Although candidates are expected to obtain a total score for Part 1 of the comprehensive EBOD examination that is equal or higher to the pass mark (pass mark equal to 6), candidates may compensate for a lower Part 1 score, provided they do not fail at any of the four stations in Part 2 of the comprehensive EBOD examination and have obtained a final EBOD score equal or higher than 6. Part 1 of the comprehensive EBOD examination represents 40 percent of the total EBOD score.

An example of an answer sheet for Part 1 of the comprehensive EBOD examination, with indication of the three possible answer options (true, false or “don’t know”) for all items, is shown in Fig. 1. An example of a modified type X MCQ item of which have been used for Part 1 of the comprehensive EBOD examination is as following. Near visual acuity might be disproportionately reduced compared with distance acuity in: (1) age-related macular degeneration (true); (2) posterior subcapsular cataract (false); (3) advanced glaucomatous optic neuropathy (true); (4) amblyopia (false); and (5) high myopia (false).

Fig. 1.

Example of the answer sheet for Part 1 of the comprehensive European Board of Ophthalmology Diploma examination where judgment is needed for each item (items displayed at the top end of the answer sheet) of any multiple-choice question (question numbers displayed at the left side of the answer sheet), whether the statement is true (T), false (F) or they don’t know the answer (D).

Candidates have the possibility to choose between three languages for Part 1 of the comprehensive EBOD examination: English (master language, which will be continued in the future), French (translation, which will be abandoned in the future), or German (translation, which will be abandoned in the future). In case candidates would have chosen for the French or German language, the questionnaire is provided only in this language without permanent availability of the items in the master language, although candidates are offered the possibility to have supervised consultation time of the master version of a specific MCQ in case of doubt.

Translation/translation verification of the master version (English) into both other languages is performed by native-speaking experts in ophthalmology. After translation meticulous check-up is performed by independent members of the EBO Examination Committee to verify the correctness of the translations, in order to assure that no (subtle) differences have been created by translation. Also, a thorough MCQ reviewing process has been installed, under the umbrella of a dedicated EBO MCQ Committee.

Part 2 of the comprehensive European Board of Ophthalmology Diploma examination

Part 2 of the comprehensive EBOD examination consists of four separate viva of 15 minutes each, which will be conducted by four panels of two examiners each. The viva will cover developmental, genetic dystrophic, degenerative, inflammatory, infectious, toxic, traumatic, neoplastic, and vascular diseases affecting the eye and its adnexa. In all panels candidates should be able to discuss preventive ophthalmology, medico-legal aspects of ocular diseases and European contributions to ophthalmology. In each panel emphasis will be placed upon the following:

(1) Data acquisition: Recognition by the candidate of abnormalities and diseases that affect the eye, ocular adnexae and the visual pathways based on clinical cases.

(2) Diagnosis: The ability of the candidate to synthesize clinical, laboratory and histopathological data in order to reach the correct (differential) diagnosis.

(3) Treatment: Candidates will be expected to provide a reasonable and appropriate plan for medical and surgical management of patients with the conditions depicted or described. No further details will be asked on details about surgical techniques proposed (no surgical skill evaluation is performed at the comprehensive EBOD examination).

Although candidates are expected to obtain a score equal or above 6 for each of the viva (pass mark equal to 6), candidates may compensate for one lower viva score, provided they did not fail at Part 1 of the comprehensive EBOD examination, and provided they did not fail at any other of the viva, and have obtained a final EBOD score equal or higher than 6. Part 2 of the comprehensive EBOD examination represents 60 percent of the total EBOD score, with an equal representation of all four viva.

Although the general language for Part 2 of the comprehensive EBOD examination is English for all candidates, they have the possibility to choose one secondary language for support in case language issues would arise during the conduct of the viva. Whenever feasible, one of the co-examiners in each jury will be fluent in the secondary language chosen by the candidates. In case it would be impossible to provide a co-examiner who is able to speak the secondary language chosen by the candidates for all viva juries, maximal efforts are exercised by the Examination Committee to ensure the presence of an interpreting ophthalmologist.

Score calculation for the comprehensive European Board of Ophthalmology Diploma examination

The comprehensive EBOD examination is composed of two different parts, with Part 1 being a written paper composed of 52 MCQs of the modified type X format and Part 2 being an oral assessment divided over 4 vivas. The overall individual (i) candidate EBOD score (EBODi) can be calculated using Formula 1.

In Formula 1 MCQi represents the converted Part 1 score on a 1–10 scale (Table 2) and VVij represents the candidate’s viva score for topic j on a 4–10 scale.

Calculation of Part 1 score of the comprehensive European Board of Ophthalmology Diploma examination

Once candidates have completed Part 1 of the comprehensive EBOD examination, all answer sheets are processed by an optical reader system, which produces a comma separated values (*.csv) output file with all exact answers of the candidates. This CSV output file is imported into Microsoft Excel to be calculated for the following parameters for all candidates by comparison of the exact answers of all candidates with the master template of correct answers: (1) Number of items that are answered only with the correct response (NC); (2) Number of items that are answered only with the incorrect response (NI); (3) Number of items that are answered only with the “don’t know” option (ND); (4) Number of items for which multiple responses are given by the candidate (NM); and (5) Number of items that have been left blank by the candidate (NB).

The sum of these parameters will always be equal to the number of responses expected to be given by the candidate (i.e. the total of test items, or 260 responses) (Formula 2).

Based on the scoring rules for Part 1 of the comprehensive EBOD examination (Table 2), the individual (i) candidate total score (CTSi) can be calculated using Formula 3.

Formula 3 can be simplified further and rewritten as Formula 4.

Although it never happened in the history of the comprehensive EBOD examination, in the unfortunate case where a candidate would obtain a negative CTSi, this negative score would be rounded up to 0. Once all final CTSi scores are available for all candidates (n represents the total number of candidates), Formula 5 and Formula 6 were used respectively to calculate the average candidate score for Part 1 (Avg) and its according standard deviation (SD), both of which can be used for conversion of the CTSi into MCQi scores (Table 2), which can be used for the overall candidate EBOD score (EBODi) by using Formula 1. As from the 2017 edition, a linear conversion scale between the minimum observed CTS (equal to 4) and the maximum observed CTS (equal to 10) will be applied.

Despite all efforts of the Examination Committee to maintain the level of difficulty at the same level over the years, it remains impossible to have exactly the same level of difficulty for each edition of the comprehensive EBOD examination (Table 3). This is the reason for the adoption of a relative pass mark system based on Formula 5 and Formula 6, in order to maintain a stable pass rate over the years (Table 3).

Table 3.

Comparison of the level of difficulty of Part 1 of the comprehensive EBOD examination indicated by the average candidate scores and the passing rate of the overall comprehensive EBOD examination from 2006 to 2016

	Year
	2006	2007	2008	2009	2010	2011	2012	2013	2014	2015	2016
Negative marking	-	-	-	-	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Average (max 260)	189	191	184	204	146	134	159	148	162	149	142
Standard deviation	14	15	15	13	25	23	24	24	23	23	22
Pass rate (%)	88.1	89.2	90.8	88.6	92.0	91.5	89.6	89.5	89.6	91.5	90.1

EBOD, European Board of Ophthalmology Diploma.

The conversion system of CTSi into Part 1 scores (MCQi) also assures that application of negative marking for incorrect answers, as has been applied since the comprehensive EBOD 2010 examination, does not decrease the chances of candidates to pass the comprehensive EBOD examination. However, application of negative marking will allow bright candidates to distinguish themselves better from borderline candidates. As a result, the standard deviation of the individual results increased starting from 2010, which is the year of introduction of negative marking for incorrect answers, which is a strong indication for the improved discriminative power of Part 1 of the comprehensive EBOD examination.

Calculation of Part 2 score of the comprehensive European Board of Ophthalmology Diploma examination

Both co-examiners agree upon a mutual score for each candidate (VVij), which can range between 4 and 10.

Monitoring and validation as measures of quality control

Pre-assessment of items

By a careful procedure of pre-selecting and controlling the quality of Part 1 of the comprehensive EBOD examination, the EBO ensures that the comprehensive EBOD examination remains an assessment in ophthalmology and not in language skills. Statistical analysis using both parametrical, non-parametrical and item-response statistics of the different language groups allowed the EBO to have objective arguments for this statement. As a matter of fact, no statistical significant differences between language groups have been observed.

The pass rate of the comprehensive EBOD examination has proven stable over the years and when compared to other European postgraduate medical assessments, the passing rate of the comprehensive EBOD examination is noticeably high. Around 90 percent of candidates pass the comprehensive EBOD examination, compared to approximately 60–70 percent of candidates for other medical specialties [3,4,10], which can be explained by the adoption of very strict eligibility criteria that are maintained for the comprehensive EBOD examination.

One of the measures taken to maintain a stable passing rate over the years is the careful pre-selection of MCQs. This not only applies to the pre-defined topics, but also involves a pre-assessment of the level of difficulty of the different MCQ-items. By this pre-assessment, the EBO aims to have comparable levels of difficulty within the comprehensive EBOD examination over the years.

Another measure taken to maintain a stable pass rate of the comprehensive EBOD examination is the application of a norm-referenced pass mark for Part 1. In order to pass Part 1, the candidate score should be above or equal to the average MCQ-score of all candidates minus one standard deviation (will evolve into a linear conversion scale as from 2017 onwards). This type of pass mark is also indicated by many educationalists according to literature on the subject, and used in other European postgraduate medical assessments.

The norm-referenced pass marking system for Part 1 of the comprehensive EBOD examination offers the advantage to candidates not to jeopardize their chance at passing, in the unlikely event that the assessment would turn out to be more difficult than had been pre-assessed by the examination committee.

In conclusion, pre-assessment of the comprehensive EBOD examination by the examination committee includes not only language, but also the level of difficulty of Part 1 of the comprehensive EBOD examination. Therefore, it can be concluded that by using this strategy, the EBO has been able to prove that no bias whatsoever has occurred over the years due to language problems or level of difficulty.

Multiple-choice questions reviewing process

The Assessment and Executive Officer of the comprehensive EBOD examination has been appointed as responsible person for the MCQ banking for the comprehensive EBOD examination, and as such MCQ authors will submit newly written MCQs to the comprehensive EBOD assessment and executive officer.

MCQs can be submitted through different origins, of which the main source is examiners participating in Part 2 of the comprehensive EBOD examination. Indeed, all examiners have to provide 3 MCQs immediately after participation at the comprehensive EBOD examination. The second major source of MCQs consists of MCQs submitted through the Accredited Courses project of the EBO, which is called the European Network for Education of Trainees (ENET). These courses are organized by European ophthalmological subspecialty societies to prepare candidates for the comprehensive EBOD examination. In order for such a course to be accredited by the EBO, one MCQ per speaker has to be provided by the organizers of the course, of which the content of the question is obviously related to the content of the course.

Detailed guidelines to MCQ authors are provided by the EBO Education Committee, of which the general architecture of MCQs consisting of one stem and 5 homogenous and independent items is of paramount importance. Besides these detailed guidelines and the examples and recommendations which are published on the EBO website, regular symposia for MCQ authors and reviewers are organized by the EBO MCQ Review Committee. In order to facilitate the implementation of the guidelines and detailed instructions by the MCQ authors, a dedicated MCQ submission form has been designed and implemented.

Once MCQs have been received by the comprehensive EBOD Assessment and Executive Officer, these MCQs will be transferred to the EBO MCQ Review Committee for primary review, which consists of identification and enhancement of incorrect and/or imprecise wording. Thereafter, secondary review of the MCQs consisting of a check on format and scientific content will be performed in EBO MCQ Subspecialty Review Committees which are organized in close collaboration between EBO and European subspecialty societies.

As soon as primary and secondary review of MCQs have been performed the comprehensive EBOD Assessment and Executive Officer will re-check the wording of the revised MCQs prior to perform translation of the MCQs from English into French and German language, which is performed by professional translators in close collaboration with a native-speaking ophthalmologist. Finally, two members of the EBO MCQ Review Committee whose native language is French and/or German will perform a careful translational review of the MCQs, after which the MCQs will be entered into the final EBO MCQ item bank, where MCQs are labelled not only with a unique identifier number, but also according to the syllabus.

Upon use of the MCQs at the comprehensive EBOD examination, the statistical performance parameters of the individual MCQs will be added as extra label into the EBO MCQ item bank. MCQs with high distinguishing power can be selected for future editions of the comprehensive EBOD examination as marker items, which will allow comparison of performance between different editions of the comprehensive EBOD examination.

Influence of negative marking for incorrect answers

In 2010, negative marking for incorrect answers was introduced for Part 1 of the comprehensive EBOD examination, as is the case for other European postgraduate medical assessments [3,4,10]. The Examination Committee would like to stress once more that application of negative marking for incorrect answers has not resulted in a decrease of candidates’ chances to be successful at the comprehensive EBOD examination. On the contrary, by reducing the number of wild guesses, candidates have been able to distinguish themselves better from borderline or minimally competent candidates. A strong indication for the improved discriminative power of Part 1 of the comprehensive EBOD examination, is the increased standard deviation of individual scores, as has been observed since the introduction of negative marking for incorrect answers (Table 3).

Primary and secondary statistical analysis methods

In addition to the pre-assessment measures described above, the Examination Committee relies on both primary and secondary statistical analysis of Part 1 of the comprehensive EBOD examination, whereas many other medical specialties only rely on primary statistical analysis for their European postgraduate medical assessments. The primary statistical analysis report is generated by the software tool used for the processing of the examination sheets (Speedwell MultiQuest, http://www.speedwellsoftware.com/). This software tool is also used for other European postgraduate medical assessments [3,4].

The primary statistical analysis consists of several parameters of which item difficulty indicated by the percentage of candidates answering correctly and item discriminative power not only indicated by the correlation of the item score with the total test score, but also indicated by a graph displaying the performance of the top 20 percent candidates compared to the bottom 20 percent candidates. This primary statistical analysis helps the Examination Committee in identifying MCQs with an optimal balance between item difficulty and item discriminative power.

The primary statistical analysis report is generated within minutes by the Speedwell MultiQuest software tool and is easy to interpret. Because of the reliability of this statistical analysis method and because of the swiftness of the process, the Education Committee is able to display the results of the examination and to organize with absolute confidence the Awards Ceremony the day after the comprehensive EBOD examination took place. The timeframe even allows for double or triple checking, as well as manual random checking.

The secondary statistical analysis tool has been created in-house by the Examination Committee and consists in an algorithm written in the statistical analysis program R. This in-house developed software tool not only replicates the methods used by the Speedwell MultiQuest software as another check of the above-mentioned analysis, but also allows a more in-depth non-parametric and Rasch statistical analysis to compare the different languages, the candidates of different countries, the residents versus the specialists, etc.

Although available within minutes, the results of this secondary statistical analysis are discussed in detail by the Examination Committee after the comprehensive EBOD examination. These results do not only help in proving the validity and stability of the comprehensive EBOD examination (validation of MCQs) but also lead to continuous improvement (e.g., the introduction of negative marking, which will be beneficial for candidates). In fact, the knowledge drawn from this statistical analysis is very useful for the pre-assessment of future examinations.

Item banking

In order to consolidate this knowledge, the Examination Committee will use dedicated software, to assure that items are not only stored in a secure manner, but also contain all knowledge derived from the items (statistical analysis results). Furthermore, this dedicated software allows history records of the modality of the items, which allows to implement new knowledge derived from statistical analysis methods to be implemented on other items within the bank. Also, a careful MCQ improvement procedure has been installed as was described earlier in this manuscript. Together, all of these efforts lead to validation.

Examiner selection procedure

All examiners involved in Part 2 of the comprehensive EBOD examination are carefully selected based on pre-defined criteria as described above.

Validation of Part 2 of the comprehensive European Board of Ophthalmology Diploma examination

Not only Part 1 of the comprehensive EBOD examination is carefully monitored and analyzed with statistical tools. This is also the case for Part 2. Examiners are pre-selected and clearly instructed on how they are supposed to assess. Over the past few years, despite the many different examiners and many different languages, it has been concluded that the results of candidates are positively correlated. Therefore, the EBO is confident to state that candidates did not experience any advantage or disadvantage by having been assigned to a specific jury.

By using the above-mentioned strategy, the EBO is proud to mention that the comprehensive EBOD examination is carefully and continuously monitored and validated, and has been shown to be reliable and stable over the years. This is the clear result of careful pre- and post-assessment of the examination.

Syllabus and recommended reading

The principles and guidelines of a curriculum for education in ophthalmology have been described by the International Council of Ophthalmology. The recommended textbooks for preparation for the comprehensive EBOD examination are the Kanski and American Academy of Ophthalmology series, which are edited in collaboration with the EBO. Also recommended are Spalton Atlas of Ophthalmology, Wills Eye Manual, and Moorfiels Manual of Ophthalmology.

Optics, refraction and visual physiology

Candidates should be familiar with the basic principles of physical and geometrical optics also used in standard optical instruments. They should understand the essentials of visual physiology including visual acuity, light and dark adaptation, accommodation, and color vision. They should know the various forms of ametropia, principles and techniques of refraction, principles of lens design, and methods of correction of ametropia including spectacles, contact lenses, intraocular lenses and refractive surgery. Candidates should be familiar with the methods for prescribing protective lenses, filter lenses, and aids for low vision.

Pediatric ophthalmology and strabismus

Candidates should be familiar with diseases affecting the eyes of infants and children and with associated systemic abnormalities. They should understand the anatomy, physiology and pathology of visual development and of the neuro-muscular mechanisms serving ocular motility and binocular vision. Candidates should be familiar with the methods of examination for the detection and assessment of sensory and ocular motor disorders. They should also know the clinical features, differential diagnosis, natural course and management of the various types of comitant and incomitant deviations. They should be familiar with the principles and the complications of surgery upon the extraocular muscles.

Neuro-ophthalmology and orbital disease

Candidates should know the anatomy of the orbit and the neuro-anatomy of the afferent and efferent visual systems. They should understand the principles of examination and investigation, including visual field testing, visual evoked responses, ultrasonography, conventional neuro-radiology imaging like conventional radiography, computed tomography, and magnetic resonance imaging scanning. Candidates should be familiar with the clinical features, pathology, differential diagnosis and management of disorders of the orbit, visual pathways, oculomotor system, and pupillomotor pathways, including the indications for, principles and complications of orbital surgery.

External diseases and ocular adnexa

Candidates should know the anatomy, embryology and physiology of the structures comprising the eyelids, lachrymal system, conjunctiva, cornea, and anterior sclera. They should demonstrate knowledge of relevant pathological processes, differential diagnosis, tests used to help in making their final diagnosis, and medical therapy as well as indications for, principles and complications of surgical procedures used in treatment of abnormalities and diseases affecting these tissues.

Anterior segment of the eye

Candidates are expected to know the anatomy, embryology, physiology, and pathology of abnormalities and diseases of the anterior chamber angle, iris, ciliary body, and lens. They will be expected to discuss conditions affecting these structures and those to be considered in their differential diagnosis. They should also be able to propose complementary tests necessary to reach the final diagnosis and to describe and provide indications for relevant medical and surgical therapy.

Posterior segment of the eye

Candidates should be familiar with the anatomy, embryology, physiology and pathology of abnormalities and diseases of the vitreous, retina, choroid, and posterior sclera. They will be expected to discuss conditions affecting these structures and those to be considered in their differential diagnosis. They should be able to propose complementary tests necessary to reach the final diagnosis and to describe and provide indications for medical and surgical therapy directed toward alleviating these conditions.

Conclusion

The comprehensive EBOD examination is one of the leading out of 38 examinations organised by UEMS European Boards and Specialist Sections from the point of number of examinees, item banking, and item contents. The item analysis is not only based on classical test theory but also on item response theory and has proven to be an appropriate tool for better item bank maintenance. From this review, future candidates participating at the comprehensive EBOD examination will get the help for their preparation. Furthermore, examinees, item providers, and supervisors of other European Boards Specialist Sections may be able to get hints for the promotion of their examinations. In the near future, efforts will be made to continuously improve the quality of assessment, which may even include adoption of additional assessment methods and techniques.