Competency-based (CanMEDS) residency training programme in radiology: systematic design procedure, curriculum and success factors.

OBJECTIVE: Based on the CanMEDS framework and the European Training Charter for Clinical Radiology a new radiology curriculum was designed in the Netherlands. Both the development process and the resulting new curriculum are presented in this paper.
METHODS: The new curriculum was developed according to four systematic design principles: discursiveness, hierarchical decomposition, systematic variation and satisficing (satisficing is different from satisfying; in this context, satisficing means searching for an acceptable solution instead of searching for an optimal solution).
RESULTS: The new curriculum is organ based with integration of radiological diagnostic techniques, comprises a uniform national common trunk followed by a 2-year subspecialisation, is competency outcome based with appropriate assessment tools and techniques, and is based on regional collaboration among radiology departments. DISCUSSION: The application of the systematic design principles proved successful in producing a new curriculum approved by all authorities. The principles led to a structured, yet flexible, development process in which creative solutions could be generated and adopters (programme directors, supervisors and residents) were highly involved. Further research is needed to empirically test the components of the new curriculum.

Introduction

Based on the Canadian Medical Education Directions for Specialists (CanMEDS) [1] framework of key competencies for medical specialists the Royal Dutch Medical Association (RDMA) decided in 2004 that postgraduate training and education for all medical specialists in the Netherlands should be reformed. In the same year, the national Board of Health Care Professions and Education (BHCPE) in the Netherlands was instituted, in which decisions on structure, innovation, quality (monitoring) and finances of all medical specialty training programmes were integrated [2]. The Netherlands was divided up into eight regions for medical education to coordinate postgraduate training in each region [3].

In 2006, the Dutch Radiology Society (DRS) assigned a national project team (Revision of Radiology Curriculum, “Herziening Opleiding RAdiologie” or HORA in Dutch) with the task of developing a new radiology training curriculum, taking into account guidelines both from the RDMA and from the European Association of Radiology (EAR), a collaboration of national scientific radiological societies in Europe, which published the European Training Charter for Clinical Radiology (ETCC), an outline for an organ-based curriculum comprising a common trunk and a subspecialty programme in 2005 [4]. The multitude of both the complex developments and the stakeholders in a variety of fields involved turned the accomplishment of the HORA project into a great challenge. We, therefore, decided to apply a systematic development approach based on evidence-based design principles. Both the development process and the resulting new curriculum [5] are presented in this paper.

Materials and methods

The Concilium Radiologicum (National Board for Education and Training in Radiology) of the DRS put together the HORA project team with experienced university and general hospital radiologists, residents and educationalists. The project team realised that the construction of a new curriculum comprised two major areas. The first major area comprises, the medical knowledge of radiology, radiological procedures and radiology organisation (the content of the new curriculum) which was based on the ETCC [4] and put together by the experienced radiologists and radiology residents in the project team. The ETCC formed a valuable starting point for multiple reasons. First of all, many authoritative and experienced radiologist from almost all national scientific radiological societies in Europe worked on this charter for years. Using these valuable insights saved time and energy. Second, the charter was officially approved by the EAR which committed the DRS. Third, using an officially approved European charter helped in gaining acceptance from the programme directors in the Netherlands for the curriculum changes. The ETCC was translated by an experienced medical translator in Dutch and corrected by the radiologists in the project team. The radiologists in the project team took into account that the increased technological possibilities of magnetic resonance imaging (MRI) and computed tomography (CT), and the implementation of picture archiving communication systems (PACS) have increased the diagnostic capabilities and efficiency of radiology departments. This increases not only the workload of the radiology department, but also the specialised knowledge required by the radiologist to effectively use these diagnostic capabilities to answer clinicians’ questions. It was assumed that future radiologists will increasingly engage in multidisciplinary collaboration with clinicians, and that interventional radiology will be a rapidly evolving field, shifting towards more minimally invasive procedures [6, 7]. The residents in the project team advocated the interests of their fellow residents and were responsible for communicating the developments in the new curriculum to them. The residents in the project team were first contact for questions of their fellow residents.

The second major area comprises, the educational outline and the toolbox of the curriculum which was provided by the Dutch version of the CanMEDS competency-based framework and the corresponding (mandatory) assessment instruments. The educationalists in the project team advised the radiologists how to translate the general framework to the context of radiology. Although different medical specialists show similarities in the competencies they are supposed to possess, important differences can occur between different specialties, however. For example, the competency “communicator” will be interpreted differently for a psychiatrist and a radiologist. For the psychiatrist, verbal communication is at the heart of the diagnostic and therapeutic repertoire, whilst digital or written communication is more important in the radiological service. Translation of the general framework to every medical specialty is therefore necessary to make the framework practical and meaningful. In this process, the educationalists gave advice on the building blocks of the curriculum, assessment strategy and instruments, quality control and implementation. Furthermore, the educationalists were responsible for coordinating the different (sub)tasks in the design of the new curriculum. Other activities of the educationalists included the organization of workshops and meetings with programme directors and the communication with the RDMA. Further educational support was available from the Dutch Advisory Board for Postgraduate Curriculum Development (DABCPD) [8] and the educational working group of the RDMA. Competency-based education is inspired by social constructivism. This modern learning theory assumes that knowledge is individually and socially constructed. Individuals give their own meaning and interpretation to the things that happen in their environment, and this meaning and interpretation is also shaped by interaction with other people. In the context of curriculum development, this means that learning should be an engaging, constructive and active process that takes place in or is easily transferred to realistic practical situations [9].

The project team organised the development process of the new curriculum around the principles of systematic design. This design approach has its fundamentals in design methodology and has repeatedly proved successful in new product development (material products as well as intangible services). Systematic design intends to stimulate creativity while, at the same time, the controllability of the design process is increased significantly. Modern systematic design embraces a large and diverse set of methods and approaches but can be characterised along four underlying principles [10]. The first principle, discursiveness, implies semi-structured guidance for the design process through a step-by-step yet iterative scheme of activities. Hierarchical decomposition refers to decomposing the design tasks into smaller, easier to handle but interdependent subtasks, setting up a so-called morphology for the design task. This step also includes the formulation of several design specifications for the whole design problem. These specifications are the requirements of the design. Systematic variation deals with the way of searching for solutions for subtasks and combining these following the morphological scheme into solutions for the complete design task. To prevent designers from endlessly searching for ever better solutions the principle of satisficing was introduced by Simon in 1969. According to this principle searching for an acceptable solution is by far superior to searching for an optimal solution [11]. Finally the solutions for the subtasks are combined into an overall solution and tested against the formulated design specifications.

Results

First, we present the features of the new curriculum and we explain how the (medical) knowledge of radiology, radiological procedures and radiology organisation, and the educational framework/toolbox of competency-based education are applied. Second, we present how we applied the principles of systematic design to the development process of the new curriculum. We provide a timeline of the development process and discuss unforeseen developments and delays the project team had to deal with during the process.

Features of the new curriculum

See Table 1 for the unique features of the new curriculum.

Table 1

Features of the new curriculum

Organ based

Uniform national 3-year common trunk, followed by a 2-year subspecialisation into one of eight directions

Competency outcome based with appropriate assessment tools and techniques

Based on regional collaboration among radiology departments, with exchange of residents between departments

Organ based

The new curriculum is organ systems based (Table 2), with integration of radiological techniques within these organ systems (Table 3). This makes it possible to anticipate the more specialised knowledge about specific diseases needed for answering clinicians’ questions.

Table 2

Organ systems modules and subspecialisations in the new curriculum

Organ systems module in the common trunk	Organ systems modules in the subspecialisation
Chest	Chest
Neuro	Neuro & head and neck
Head and neck
Musculoskeletal	Musculoskeletal
Cardiac	Cardiac
Gastrointestinal	Abdominal (gastrointestinal, urogenital & gynaecology)
Urogenital
Paediatric	Paediatric
Breast	Breast
Interventional	Interventional

Table 3

Radiological techniques in the new curriculum

Conventional radiology

Ultrasound

Computed tomography

Magnetic resonance imaging

Biopsies

Intervention

Nuclear diagnostics

Common trunk and subspecialisation

The common trunk encompasses a uniform national training programme on the basics of all organ systems (Table 2). The subspecialisation programme in the last 2 years of training focuses on one of eight subspecialisations (Table 2). There are a number of reasons to implement the common trunk and subspecialisation structure. First, the ETCC adopted this model. Second, this structure anticipates the rapidly growing knowledge base. It seems impossible to learn and keep up to date will all relevant knowledge in radiology. The total knowledge base is therefore divided into eight subspecialisations. This allows individual radiologists to become experts in and keep up to date with a specific subspecialisation, whilst the aggregated individual competencies in the team of radiologists comprise the total knowledge base needed. Third, the structure gives residents the motivational opportunity to distinguish themselves from their colleagues. Fourth, the new structure formalises and recognises expertise of different radiology departments. Because of differences in patient populations and existing investments, radiology departments differ in the procedures they perform and the knowledge bases they possess. Departments can enhance their expertise by teaching the associated subspecialisation. Finally, the structure improves the match of supply and demand of labour. The inflow of radiology residents into subspecialisations can be adjusted to the number of radiologists with particular profiles required in the field.

The resident can specialise in one subspecialisation or continue general radiology training in the last 2 years. Fifty per cent of the time in the last 2 years is devoted to the subspecialisation. The other 50% is devoted to general radiology. The subspecialisation is open to residents who perform above average. In addition to general and specialised radiology training, the new curriculum encourages residents to pursue scientific training. This consists of 1 day each week devoted to science, equalling 20% of total training time. This training option is open for residents who have conducted PhD research or are about to finish one.

Competency outcome based

Based on the seven CanMEDS core competencies, a specific competency profile for radiology was put together [12]. For example, the general competency “communication” was subdivided into “communication with clinicians” and “communication with patients and family”. Within “communication with clinicians”, specific competencies such as “the resident reports relevant, accurate and explicit findings of radiological diagnostics timely” were formulated.

The organ systems comprise the themes, or basic building blocks, of the curriculum. For each organ system, competency levels (Table 4) were assigned for year one, year three and year five [13]. For year five, two competency levels were assigned for each subspecialisation; one competency level for residents who follow the specific subspecialisation and one competency level for residents who follow general training.

Table 4

Competency levels in the new curriculum [13]

Level 1: knowledge possession

Level 2: performing with high supervision by radiologist

Level 3: performing with moderate supervision by radiologist

Level 4: performing without supervision by radiologist

Level 5: supervising and educating during the performance

For each of the organ system modules, three entrustable professional activities (EPAs) were composed. These are information-rich activities which assist residents and radiologists in focusing educational attention [14]. CanMEDS competencies were attributed to each of the organ system modules and for each of the EPAs. For example, within the organ systems module “Cardiac”, the EPA “patient with signs of angina pectoris as interpreted from the CT/MRI” was formulated. The CanMEDS competency “collaboration” and “professional” were attributed to this EPA, because fast and efficient consultation with the cardiologist is required.

In the new curriculum, multiple assessment instruments are being used (Table 5) to assess progress of the residents on the competency levels for the organ systems modules and the EPAs. Multiple assessment instruments are necessary to draw reliable and valid conclusions [15]. For each assessment instrument, competencies were assigned and frequency requisites formulated (Table 5).

Table 5

Assessment blueprint (instruments, accompanying competencies and frequency of use)

Assessment instrument	Competency	Frequency of use
Mini-clinical evaluation exercise (mini-CEX)	Medical expert, communicator, collaborator, scholar, manager, health advocate and professional	Ten observations each year (obligatory)
Multi-source feedback	Medical expert, communicator, collaborator, manager and professional	Once each year (not obligatory)
Objective structured assessment of technical skills (OSATS)	Medical expert, communicator and professional	Ten observations each year (not obligatory)
Critical appraised topic (CAT)	Medical expert, communicator and scholar	Twice each year (obligatory)
Progression tests	Medical expert, scholar and health advocate	Twice each year (obligatory)
Portfolio and periodical meetings with programme director	Medical expert, communicator, collaborator, scholar, manager, health advocate and professional	1st year, five meetings; 2nd and 3rd year, three meetings
		4th and 5th year, two meetings

Assessment can be formative (guiding future learning, promoting reflection) or summative (making an overall judgement about competence, qualification for higher levels of responsibility) [15]. The assumptions of the assessment strategy of the new curriculum include:

Assessment is mainly formative.

Summative assessment takes place at the end of each year of training. Multiple observations in varying settings from different assessment instruments are necessary to make a summative judgement about overall competence. Single observations from one assessment instrument are insufficient for sound judgement of competence.

The assessment instruments assess all competencies in every year of training.

Residents need to master the competencies on the required competency levels.

Premature termination of the training is possible when the assessment results indicate (persisting) disfunctioning on at least two competencies.

The team of supervising radiologists are responsible for the assessment.

Use of assessment instruments and providing feedback requires training for both the supervisors and the residents.

The mini-CEX is a method of assessing competencies in real-life clinical practice. It consists of a short observation by a qualified medical specialist of a resident demonstrating clinical skills, using a predefined scoring format, in congruence with the competency profile of the radiologist, followed by a structured feedback conversation [16]. Assessment in the feedback conversation is formative. When feedback is brought safely, timely, specific and well structured, this assessment can be a valuable mechanism of supervision and learning [17].

Multi-source feedback is a method of gaining feedback from multiple people with different functions in the radiology department [18]. This method is intended to be used as input for annual progress assessment interviews between resident and programme director.

The objective structured assessment of technical skills (OSATS) is a method of assessing technical skills and is particularly suitable in the operation theatre [19], or, for radiology, in the intervention room.

The critically appraised topic (CAT) consists of a systematic literature search on a specific structured clinical question [20].

A national knowledge progression test is being applied annually to demonstrate progress throughout the training period and allow group comparisons.

In their portfolios, residents collect evidence about their personal development. Results of assessments are recorded, and evidence of acquired knowledge and skills is collected, including repeated reflection on acquired competencies, competency levels, and a personal strengths–weaknesses analysis [8]. The portfolio is reviewed in the annual progress assessment interviews.

All assessment instruments were adapted for use in radiology and after adaptation not assessed for validity and reliability. However, the psychometric properties of every single instrument was tested in other settings. Epstein provides an excellent overview of the properties of many assessment instruments [15]. For example, research has shown between seven and eleven different observations with the mini-CEX are necessary to achieve a generalizable global estimate of competence [15, 21].

Regional collaboration

Historically, the entire radiology residency training was provided at a single medical centre. Although some radiology departments and hospitals made agreements on multi-centre training programmes and regional collaboration in education, this was not formalised in the national curriculum or legislation. The new curriculum recognises that training can benefit from alternating training periods in both university and general hospitals, and from an active collaboration between centres in regional educational activities. The national curriculum and educational framework can be used in different ways in every region of medical education, but the exchange of residents within a region is mandatory. Furthermore, every region is required to make agreements on which subspecialisations in the last 2 years of the curriculum they aspire to offer.

The development process of the new curriculum

Discursiveness

Figure 1 illustrates the design and implementation process. The design process was largely specified beforehand and divided into several design steps. The process was highly iterative. Because of developments in later phases prototypes were refined and improved.

Fig. 1

Timeline of the design and implementation process of the new curriculum

Hierarchical decomposition

From the start of the project, the HORA project team realised that the support of the radiology programme directors was a prerequisite for the new curriculum to be accepted. For this reason, the HORA project team organised three plenary meetings (Fig. 1) involving all programme directors. For the first plenary meeting, several design specifications of the new curriculum formulated by the HORA project team (see Table 6) were discussed with the programme directors, and agreed upon by all but one.

Table 6

Design specifications and accompanying discussion points for the new curriculum

Design specification	Discussion point
1. A new curriculum needs to be designed	No discussion
2. The new curriculum needs to encompass training periods in university and general hospitals	Some programme directors were concerned about reduction of autonomy due the training in both university and non-university hospitals
3. The exchange of residents between university and general hospitals needs to be organised within the eight regions of education in the Netherlands	Some programme directors were concerned about reduction of autonomy
4. The new curriculum needs to be competency based and modularly designed	Some programme directors were not convinced of the advantages of competency-based education and were worried about the additional time needed for using the new instruments
5. The new curriculum needs to consists of a 3-year common trunk and 2-year subspecialisation	Some programme directors were afraid that insufficient radiologists with a general profile would be trained
6. The new curriculum needs to be organ based	Some programme directors were concerned about the amount of time and energy necessary for organising the rooms, equipment, teams and ICT infrastructure differently

For the second plenary meeting, all programme directors were asked to discuss the specifications of the new curriculum within their region of medical education, and to come up with further specifications and discussion points for every specification. The second meeting made clear that all eight regions of medical education supported the specifications, and several useful discussion points were put forward (Table 6). For the third plenary meeting the HORA project team broke down the design task into smaller, but interdependent subtasks according to the guidelines provided by DABPCD and the working group RDMA (see Table 7 for the morphological scheme of the design task).

Table 7

Morphological chart: subtasks and solutions in the new curriculum

Design task	Subtasks	Possible solutions		Time period of decisions	Primary responsibility	Solution
Design of new curriculum radiology	1. Profession analysis/radiology specific profile	The general CanMEDS frameworka, the European Training Charter [4] and the Good Practice Guide For European Radiologists [12]		Short	Project team HORA	For each competency several radiology-specific subcompetencies were formulated
	2. Themes/building blocks	The organ based systems provided by the European Training Charter [4]		Short	Project team HORA	See Table 1 (left side)
	3. Assessment strategy and instruments	Assessment strategy can be formative (oriented towards formulating learning points) or summative (making a judgement about competence). Many assessment methods are possible [20]. For example:		Short	Project team HORA	See Table 3
		- Multiple choice questions/progression test
		- Essay
		- Short answer question
		- Oral exam
		- Objective structured clinical examination
		- In-training evaluation report (e.g. mini-CEX)
		- Multi-source feedback
		- Critical appraised topic
		- Portfolios and logbooks
	4. Teaching philosophy	Competency-based educationa		Short	Project team HORA	Competency-based education
	5. Teaching activities	Many teaching activities are possible. For example:		Short	Project team HORA	- Radiological workstation
		- Patient reporting				- Consultations
		- Radiological workstation				- Multidisciplinary conferences
		- Cursory conferences				- Cursory conferences
		- Journal clubs				- Structured series of conferences
		- Critical appraised topic				- Critical appraised topic/“holy hour”
		- Teaching round				- Intervention room/operating room
		- Outpatient clinic
		- Patient round
		- Intervention room/operating room
		- Consultations
		- Mortality/morbidity review
		- Multidisciplinary conferences
		- Structured series of conferences
	6. Teaching materials	Many teaching materials are possible and need to be made. For example:		Short and medium	Project team HORA for the short term and the regions of medical education for the medium term	The curriculum was written with chapters corresponding to the subtasks in this table. Teaching materials for the assessment instruments were developed (see Table 3). Other teaching materials were or have to be developed by the regions of medical education
		- Curriculum/training programme
		- Assessment instruments
		- Course materials
		- Guidelines/protocols
		- Literature
	7. Training/curriculum structure	Curriculum structure		Short	Project team HORA	3-year common trunk and 2-year subspecialisation. In the subspecialisation phase 50% of the time is devoted to general radiology and 50% to the chosen subspecialisation
		Possible solutions:
		- General training
		- Subspecialised training
		- Combination of general training and subspecialised training
		Collaboration with other radiology departments for the exchange of residents	- Collaboration with radiology departments within a region of medical education or between regions of education	Short	Project team HORA	Collaboration and exchange of residents takes place among the radiology departments within the regions of education. Exceptions can be made for individual residents
			- Duration of exchange periods; between 6 months and 4 years	Short	Project team HORA	Minimum of 1 year
			- Possibility of unconnected exchange periods e.g. two periods of 6 months	Short	Project team HORA	The exchange can be unconnected
			- Year of training in which the exchange has to occur; year one to year five	Medium	Region of medical education	Variable
			- Labour law and appointments; separate appointments for every hospital or one appointment with the hospital in which the main part of training is followed	Medium	Region of medical education	Variable, predominantly by one appointment
			- Financing of residents, by the hospital in which the resident follows the training at the time or by the hospital in which the residents follows the main part of training	Medium	Region of medical education	Variable, predominantly by the hospitals in which the resident follows the main part of training
	8. Quality monitoring	Requirements for hospitals to offer the common trunk and to offer subspecialisations		Long	National subspecialisations sections for radiology specific requirements. RDMA for more general requirements	Formulated for every subspecialisation
		Requirements for individual supervisors		Long	RDMA	Formulated for all medical specialists
		Quality control mechanisms to monitor curriculum implementation. Possible mechanisms include:		Medium	Project team HORA/Concilum Radiologicum	HORA 2 project team was installed to monitor the implementation of the new curriculum. The project team consists of one representative of every region of education and one educationalist. Further mechanisms which are used include:
		- New project team to coordinate and monitor the curriculum implementation				- Official quality audits/site visits
		- Periodical reporting by programme directors				- Periodical reporting by the representative of every region of education
		- Official and non-official quality audits/site visits by independent observers				- Digital questionnaire to all radiologists and residents
		- Questionnaire to programme directors, radiologists and residents

aMandatory by the RDMA, possible solutions were limited

Systematic variation

For each subtask, several solutions were formulated (Table 7). For some subtasks the solutions were limited by legislation. For example, competency-based education was chosen by the RDMA as the teaching philosophy for all Dutch medical postgraduate training.

Satisficing

After the solutions were formulated, the HORA project team made a distinction between solutions that had to be solved in the short (within months), medium (within 2 years) and long term (within 5 years), and between decisions that ought to be made by the project team HORA or by other parties (see Table 7). The decisions that had to be made in the short term were prepared by the HORA project team and discussed in the third plenary meeting with all programme directors. Agreement could be reached on all short-term decisions (see Table 7). Many decisions were delegated to the regions of medical education to allow for optimal alignment to the local context, and to avoid information overload of the HORA project team. For each subspecialisation, a dedicated section of DRS members formulated quality criteria for the radiology department offering subspecialisations, along with an approval procedure for radiology departments, and a registration system for residents and radiologists who successfully complete a subspecialisation training programme.

Finally the total design of the curriculum (Table 7) needs to be tested against the design specifications (Table 6). All design specifications were met.

Unforeseen developments and approvals

After the three plenary sessions, the curriculum was completed pending approval by several authorities (Fig. 1). After approval by the Concilium Radiologicum, the curriculum was evaluated and approved by the DABPCD. The DRS approved the new curriculum in a general members meeting. Before formally approving the new curriculum, the RDMA decided to introduce their own educational advisory group (working group RDMA) to give educational approval. Although this working group approved the curriculum this caused a 1-year delay. Subsequently, the curriculum had to be formally approved by the RDMA. Because of a high workload (other medical specialties were also completing their new curricula), the RDMA approval procedure caused another 1-year delay. The final step was approval by the Dutch Minister of Healthcare. To avoid further delay, the working group RDMA advised the HORA project to start implementation in September 2008, whilst formal approval of the RDMA and the Dutch Minister of Healthcare was still pending. For this reason, the development and implementation process overlapped for about 15 months. The RDMA approved the curriculum in June 2009.

Discussion

We have described the development procedure and main components of the new radiology curriculum in the Netherlands. Only when the new curriculum has been in use for a number of years will we be able to reflect on the components of the curriculum and its impact on the performance of young radiologists and the organisation of radiology departments. Here, therefore, we will discuss the systematic development process.

The highly dynamic and non-linear development process was influenced by many stakeholders, developments and unforeseen factors. Although this is the rule rather than the exception in such innovation and change projects [22], it is possible to discuss the usefulness of the systematic design procedures, and to identify success factors in this project of complex change.

Discursiveness

The design process was largely specified beforehand in a logical step-by-step iterative approach. The specification beforehand proved valuable because it was clear to everyone what steps needed to be taken and which results needed to be achieved for particular meetings. Another strength of the discursiveness principle is that unforeseen developments can be incorporated. This seemed useful, especially during the curriculum approval phase. Despite the 2-year delay caused by an additional approval step and the high workload of the regulatory authorities, these steps significantly improved the quality of the curriculum.

Hierarchical decomposition

The HORA project team broke down the overall design problem into several subtasks. For the overall design six design specifications were formulated, and these were discussed with the programme directors. These steps proved extremely useful. First, it made the complex design task easier. Second, involving the programme directors from the start of the process in deciding on the specifications of the new curriculum made them responsible for the ultimate results and prevented them from re-discussing the specifications and the necessity of curriculum change in a later phase.

Systematic variation

The HORA project team, the regions of medical education, and the programme directors came up with several solutions for the subtasks, which allowed for an efficient division of tasks among individual members and subgroups, avoided information overload and involved the relevant stakeholders.

Satisficing

Making a distinction between short-, medium- and long-term decisions helped to focus the debate on decisions that needed to be taken at a specific moment in time. By outsourcing and decentralising some decisions, people were able to adapt these to local circumstances, and information overload of the HORA project team was avoided. It has been shown that innovations are adopted better and more quickly when adopters can tailor the innovations to their own needs [23].

Overall, the use of systematic design principles to structure the development process proved highly valuable. They led to a structured, yet flexible, development process in which creative solutions could be generated and adopters (programme directors, supervisors and residents) were highly involved. This is a prerequisite for the adoption and successful implementation of the new curriculum.

Strengths, limitations and suggestions for further research

The description of the design (content) of the new curriculum, as well as the development process, adds to the existing knowledge base and could be useful for scientific communities of radiology and programme directors who are about to start or are currently busy with designing a new curriculum. Several papers have appeared on curriculum design and evaluation of the CanMEDS framework within medical education [24-28] and radiology [29], but the integral description of the design and of the development process according to evidence-based systematic design principles has not been documented before.

There are some limitations. First, the paper lacks empirical data on the implementation and ultimately the effectiveness of the design of the curriculum and the application of design principles. It would be interesting to empirically test the implementation and the effectiveness of the subtasks of the design (Table 7), for example the organ systems-based modules (Does it improve the quality of radiological service? What is the influence on the quantity of requests for radiological research? Is there any change in the subjective perception of the radiologists of their jobs? What is the influence on effective manpower management?). It would also be interesting to empirically test the importance of every principle in such complex innovation processes. Second, the paper is limited to the Netherlands. Because of differences in patient populations and the organisation of radiology departments the curriculum probably needs some adjustments before it can be implemented in other countries.

Conclusions

The HORA project team incorporated many complex developments and succeeded in producing an exciting new radiology curriculum in the Netherlands. This curriculum is necessary for the future of radiology in the Netherlands, providing not only high quality training for radiology residents, but also advancing the science of radiology in the next few decades. The use of the systematic design principles proved useful in the development process of the new curriculum. The next challenge is to implement the curriculum in everyday practice.