HERMES European Accreditation of Training Centres in Adult Respiratory Medicine: criteria validation and revision.

Four respiratory medicine disease categories appear in the global top 10 causes of mortality [1], resulting in 600 000 people dying from respiratory disease in Europe each year. The economic burden of respiratory diseases in Europe exceeds 380 billion euros. In a fast-developing environment, new clinical challenges have arisen for pulmonary specialists; techniques and procedures have evolved and become more complex.

Introduction

Respiratory medicine and training centre accreditation

Four respiratory medicine disease categories appear in the global top 10 causes of mortality [1], resulting in 600 000 people dying from respiratory disease in Europe each year. The economic burden of respiratory diseases in Europe exceeds 380 billion euros. In a fast-developing environment, new clinical challenges have arisen for pulmonary specialists; techniques and procedures have evolved and become more complex.

Consequently, the work of health professionals has radically changed in the past two decades, resulting in an increase in the number of respiratory specialists [1]. Increased specialisation and the fact that most respiratory departments now have well-equipped pulmonary function laboratories has led to the creation of multi-disciplinary teams in which respiratory physicians must collaborate.

Meanwhile, the European Parliament encourages the mobility of European citizens across its member states, yet the academic and training infrastructures as well as the education systems still greatly vary across Europe. These differences depend on a nation’s economy, societal influence, medical history and traditions.

The European Respiratory Society (ERS) and European Board for Accreditation in Pneumology (EBAP) believe that the harmonisation of training for respiratory specialists and the accreditation of training centres are paving the way to better patient care across Europe.

Medical training accreditation worldwide

To date, the Foundation for Advancement of International Medical Education and Research (FAIMER) has identified 114 countries [2] with active accreditation or recognition systems for basic medical training. Accreditation systems, criteria and accreditation bodies’ remits vary across countries. Accreditation bodies can be directly linked to governmental institutions or health ministries or be private accreditation bodies that are mandated to conduct the audits. Some countries have several accreditation bodies and accreditation of training can either be compulsory or ­voluntary. A benchmark performed by the ERS of other specialty training accreditation programmes suggest the same applies to the accreditation of medical specialty training.

Although little research has been published demonstrating the direct impact of training accreditation on patient care, it is safe to say that training centres applying for accreditation will review their education and assessments methods [3]. They will also ensure that the appropriate protocols are in place to comply with the defined accreditation standards. We can therefore assume that the accreditation process may encourage training centres to implement up-to-date educational and assessment standards.

A recent study regarding the impact of accreditation on the workplace educational climate in gynaecological oncology across Europe [4] demonstrated that institutions accredited for sub-­specialty training in gynaecology tend to provide better training environment with respect to supervision, coaching, assessments, feedback, teamwork and peer collaboration. The same study showed that education at accredited centres was better structured/formatted, resulting in a higher trainee satisfaction level [4].

Other studies performed in the USA [5, 6] suggest that the accreditation of training programmes has a positive impact on trainees’ pass rate during some examinations and demonstrated that, overall, trainees who have attended accredited training programmes generally perform better.

Overall, the general perception is that formally accredited medical institutions demonstrate a certain quality level [7], which presumably will influence learners in their choice of training centre, increasing the institution’s reputation and ultimately generating revenues for the accredited departments.

It is the ERS and EBAP’s intention that this voluntary accreditation programme will help centres achieve higher quality training levels by benchmarking themselves against other European Training Centres.

The Accreditation Committee hopes that this initiative will motivate directors and faculty members to further develop and improve the training, educational and assessment methods within their departments.

Adult HERMES and accreditation project

Since 2005, nine Harmonised Education in Respiratory Medicine for European Specialists (HERMES) task forces have been initiated, with the aim to standardise training and education within different sub-specialties of respiratory medicine. The HERMES Adult Respiratory Medicine was the first HERMES project launched to promote harmonised education and training. It is divided into five phases (figure 1).

Figure 1

The five phases of the HERMES projects.

Various partners play an active role in the accreditation of training centre process, including the ERS that defined the accreditation criteria.

Following the publication of the criteria for accreditation of ERS European training centres in adult respiratory medicine in Breathe in 2010 [8], the EBAP joined the initiative. Its Management Council agreed that EBAP could actively partake in this programme and lead the review process. Stakeholder roles and responsibilities are outlined in figure 2.

Figure 2

Training Centre Accreditation stakeholder roles and responsibilities.

Whenever possible, a national reviewer, preferably someone involved in the national accreditation process, is included in the review team. This ensures that the local regulations, socio-economic context and customs are taken into account by the review team.

European Training Centre Accreditation

Method

The HERMES Adult Respiratory Medicine Task Force began designing the Training Centre Accreditation Criteria in September 2009. Initially, the Task Force researched the field and used a number of published documents as a basis for the criteria document structure.

To ensure that all aspects of specialist training were covered, criteria were developed by the Task Force in collaboration with a wider group of experts including clinical/educational supervisors, trainers and trainees in Adult Respiratory Medicine.

The task force sought input from health professionals across Europe by means of a survey. Respondents specifically indicated the prescribed numbers of procedures trainees should be exposed to in order to attain knowledge necessary for independent specialist practice.

Results of the surveys were analysed and discussed during a workshop in March 2010. Criteria were developed between September 2009 and June 2010, eventually leading to the publishing of the criteria for accreditation of ERS European training centres in adult respiratory medicine in Breathe in December 2010 [8].

Process development

In 2011, ERS decided to pursue the accreditation process. It was however agreed that the ERS, as a scientific society, could not legitimately grant accreditation. The ERS Education Council came to the conclusion that an official accreditation body should be involved to ensure fairness, impartiality and to legitimize the process.

Naturally EBAP, as the European pneumology accreditation body, was sought to become the primary partner in this project. The EBAP Management Council accepted to join forces with the ERS and ultimately to lead the review and accreditation processes. The Accreditation Committee has also begun to seek partnership with other European accreditation bodies.

In order to develop the accreditation process and related documents, a joint ERS/EBAP working group was initiated and began work in 2012.

Following recommendations by the World Federation of Medical Education (WFME) [9], criteria were divided in two levels:All basic standards must be met to ensure minimum quality in training delivery. Quality development standards are desirable but not compulsory [10]. It is however expected that training centres will continuously improve their processes and eventually meet all standards at both levels.

Basic standards

Quality development standards

The ERS benchmarked themselves against other existing specialty training centre accreditation processes (Accreditation Council for Graduate Medical Education (USA); European Association of Neurosurgical Societies; European Board of Neurology, European Board of Anesthesiology; European Board of Urology and European Society of Radiology) to determine best practice. The benchmark, backed by literature [11, 12], led to the current two-fold process comprising a self-reporting or qualification phase followed by a site visit (figure 3).

Site visit to HELIOS Lungenklinik in Berlin, Germany, December 2013.

The working group then started developing a series of documents supporting this two-step process, including an application form and questionnaires for the various stakeholders at the training centres. A challenging aspect of the development process was to create comprehensive documents and forms that would ensure that each accreditation criterion could be properly assessed by the review team. The Accreditation Committee is confident that the updated criteria are suitable for application in real-life settings Europe-wide.

Pilot phase and revised criteria

In winter 2013–2014, the ERS/EBAP Accreditation Committee was ready to pilot the process. Two training centres were selected for their differences in size and geographical regions.

The HELIOS-Klinikum Emil von Behring Lungenklinik Heckeshorn, Abteilung für Pneumologie in Berlin, Germany, and the Semmelweis University Clinical Center, Department of Pulmonology in Budapest, Hungary, went through the process successfully and were granted accreditation in spring 2014.

Feedback from the pilot centres and observations from the review teams were used to improve the supporting documents and processes. These suggestions were also taken into account when revising the accreditation criteria.

Criteria re-validation and revision

The criteria re-validation and revision process began in August 2014. An online survey was sent to a group of experts independent of the initial task force members and national respondents. This new group comprised of members from the Accreditation Committee, Review Committee as well as EBAP reviewers.

A total of 20 professionals from 18 European countries completed the survey. The surveyed professionals mainly had roles as professors or associate professors in respiratory medicine, heads of department in public or private institutions and senior consultants in respiratory medicine.

The respondents indicated, for each criterion, if they agreed with the categorisation as a “basic standard” or as a “quality development standard”. All 56 criteria were validated by the respondents, with an agreement rate of >90%. Those respondents who expressed concerns thought it would be difficult to adapt specific criteria to their national regulations or training specificities.

Areas of concerns mainly included the recommended number of trainee full time equivalent positions, pre-requisite number of years in general internal medicine training, recommended number of patient exposure at in- and out-patient services, prescribed exposure to specific procedures as well as education/assessment methods; all quality development standards.

A few comments and suggestions were made regarding basic standards, such as the prescription for elective training, the required number of publications and the support facilities (clerical support, availability of room and catering during night-duties, etc.).

Between winter 2014 and summer 2015, the Accreditation Committee discussed each criterion, especially those flagged as areas of concerns, and amended them accordingly.

In parallel, the Training Centre Accreditation in Paediatric Respiratory Medicine Working Group also began drafting accreditation criteria relevant to paediatric respiratory medicine. In order to streamline both accreditation documents, comments from the paediatric working group were also taken into account where appropriate and applicable to adult respiratory medicine.

Criteria pertinent to patient involvement [13, 14] were included in the document following recommendations from the European Lung Foundation.

Finally, the support from several educationalists (ERS staff and external consultants) was key to enhancing the document and, in particular, criteria related to the content of educational experience, learning environment [15], and educational and assessment methods.

Challenges

One of the biggest challenges when designing and revising the criteria and related processes was to account for the different medical education and accreditation systems in place throughout Europe [11].

This was overcome with the help of the national respondents through the surveys conducted, and also by adapting some key criteria, referring to European Union and/or local regulations.

Committees

In order to ensure fairness of the review and accreditation processes, two joint ERS/EBAP committees were established. Members of both Committees are appointed for a non-renewable term of 3 years, with the exception of the ERS Education Council Chair and EBAP President, whose terms are aligned with the duration of their mandates. The structure of the review and accreditation committees is shown in table 1.

Table 1

Structure of the Review and Accreditation Committees.

Review CommitteeComposition Reviewers nominated by ERS and EBAP, including representatives from accredited centres. ERS and EBAP each nominate a Review Committee Co-Chair.Responsibilities Co-Chairs appoint two reviewers to each application, one EBAP and one ERS representative. Reviewers  Assess the application based on the “Adult HERMES: criteria for accreditation of ERS European training centres in adult respiratory medicine” and provide recommendations to the Accreditation Committee.  Perform a 2-day site visit incorporating interviews, and make recommendations to the Accreditation Committee.  Prepare a report to the training centre after each step of the process (qualification and site visit).Accreditation Committee Composition Co-chaired by the ERS Education Council Chair and EBAP President. Composed of the Review Committee Co-Chairs, a representative from previous site visits and a Junior Committee representative. Members are appointed for a non-renewable term of 3 years, with the exception of the ERS Education Council Chair and EBAP President whose terms are aligned with their office term. Responsibilities Grant accreditation based on review team recommendation. Review accreditation criteria. Implement guidelines and recommendations. Oversee all accreditation processes for respiratory medicine, including at sub-specialty levels.

A reviewer’s perspective: Interview with Professor Johan Verbraecken, Training Centre Accreditation Review Committee Co-Chair

How exactly are you involved in the training centre accreditation process?

At the end of my mandate as ERS E-learning Director in 2013, I was invited by the ERS Office to participate in a new project regarding the accreditation of respiratory medicine training centres. In the beginning, this area was brand-new to me, but fortunately, a lot of preparatory work had already been completed by a working group of representatives from the ERS and EBAP.

In July 2013, I underwent training in Lausanne, Switzerland, together with Kostas Kostikas, to get familiar with the review procedure and application process. After formal application by a candidate centre, the documents must be reviewed and checked to ensure that they fulfil the standards (basic and quality development). Also, the number of procedures which the training centre has to perform annually to ensure that the trainees’ attain adequate exposure is reviewed. Once a centre qualifies for accreditation, a formal visit is organised in order to check the effective quality of the training centre. Given the workload that this will entail in the forthcoming years, due to the many applications received, the committee has decided to expand the team and to reschedule the duties.

The experience we had with two pilot hospitals helped us to optimise the criteria and to finalise the format of the application manual. The current Training Centre Accreditation committee (­consisting of Gernot Rohde (ERS Education Council Chair), Daiana Stolz (EBAP President), Johan Verbraecken (Review Committee; ERS Co-Chair), Ortrud Karg (Review Committee; EBAP Co-Chair), Szymon Skoczyński (Junior Member Committee Representative) and two staff members (Sandy Sutter for EBAP, Sharon Mitchell for ERS)) meets twice a year and also has teleconferences. The role of the Review Committee Co-Chairs is to be the link between the Accreditation and Review Committees. A similar process is currently ongoing for accreditation of ERS European training centres in paediatric respiratory medicine.

Could you tell us a little more on a reviewer’s duties for this process?

Step one of the accreditation process is for the centre to provide the designated accrediting body with a number of documents, in order for the reviewers to ensure that the centre complies with all basic standards, as described in the accreditation criteria. This extensive application form, includes data on the following.Also, a number of additional documents are ­mandatory for review, including:The first step is a thorough examination of these documents, firstly by the ERS Office, to ensure the information provided is complete, then by the Chairs of the Review Committee (myself and Ortrud Karg). The answers are also screened for inconsistencies and reliability. All mandatory elements have to be fulfilled before a training centre qualifies and is ready for a site visit. Conclusions of the committee are shared or discussed electronically or during a face-to-face meeting. In case of uncertainties, the candidate centre is contacted for additional information. Once the committee agrees, a report is then issued to the training centre, highlighting areas of improvement. Only centres complying with all basic standards as defined in the Accreditation Criteria document will then be able to undertake the next step of a site visit.

Name of training centre

If training centre is part of a network and data on any partner training centres

Programme director

Teaching faculty

Educational supervisor

Clinical supervisor

Trainees

Training programme, including curriculum and rotation plan

Detailed information linked to the accreditation criteria

Programme director curriculum vitae and list of 10 most recent publications in English

Clinical/educational supervisor curricula vitae and list of 10 most recent publications in English

Comparison of training centre and HERMES syllabi

Full curriculum in English

Education and clinical programme (weekly timetables) in English

Trainee rotation plan in English

List and contact details of people involved in the visual inspection, including teaching ­faculty, clinical/educational supervisor(s) and trainees

You have been involved in several reviews and site visits, could you tell us which are, in your opinion, the most difficult criteria or aspects of training to assess as a reviewer?

The documents to be provided by the training centre before the site visit are used as the basis for the interviews conducted onsite by the review team. The reviewers are really dependent on the statistics about in- and out-patient activity offered by the institute. As a reviewer, I try to get a confirmation of these data by interviewing different sources and performing visits to the different units of a department. Also, internal evaluation processes need special focus, since these activities are often less formally organised than centres report in their application form. Completeness of protocol books and procedures is also an aspect that needs particular attention. Finally, the degree of involvement in research during the training is hard to monitor and to quantify, and is based on subjective reporting.

What was your perception of the site visits you performed? what were the most difficult or challenging aspects of the site visits?

I was honoured to have visited two very well administrated hospitals, which were well staffed and structured. Both services were very well prepared for the site audit, and spent plenty of time and energy to support the audit. We conducted not only focused interviews with the programme director, teaching faculty and educational/clinical supervisor, but also had open discussions with staff members, as well as trainees and nurses. All levels were highly motivated to receive the certification. Since I had been ambassador for Joint Commission International accreditation in my own hospital (Antwerp University Hospital) for the last 3 years, I was already familiar with these kinds of audits, which led me to have attention for more specific quality criteria. Therefore, I was sometimes more strict than my colleagues auditors. One observation was the difference in infrastructure (i.e. equipment, buildings) observed during the different visits, but we learned that excellent training can also be offered in old-­fashioned facilities.

What do you think this process brings to the training centres and what was the feedback from the training centre personnel when this was piloted?

I’m quite sure that both centres improved the quality of their training programme before the application was submitted, in order to reach the standards. During the audits, we also observed some quality improvement items that will help them to optimise their programme. Anyway, these centres were extremely grateful that they received the certification, which is a quality label and a reward for the high level of training they offer. Pilot centres found the application procedure demanding and time consuming, and these comments were taken to the Accreditation Committee. Meanwhile, some adaptations have been made to the application manual.

In addition, the assessment might facilitate improvements in infrastructure or in the current needs of the training centre in discussion with the hospital or academic leadership. In the future, when a sufficient number of centres has been accredited, this will also enable benchmarking of training centres against European standards.

What did taking part in this whole process and acting as a reviewer bring to you? Have you learned anything through it?

During the different audits, I learned that there are huge differences in hospital facilities in West and East Europe. However, this is not necessarily translated into better care for patients and better training for trainees. Collaboration, protocols, guidelines and supervision are key for a successful training programme. I also got ideas for my own hospital organisation.

Finally, would you recommend that training centres go through the process and why?

I absolutely recommend training centres that use high standards to apply for certification. The process provides a drive to review their processes and the structure of their training, and helps them to have more insight into the organisation and will even stimulate them to fill the gaps. Once the accreditation label is received and communicated to the community, it will attract new trainees and even scientists to these centres of excellence, and will enable exchange with trainees from other countries. Since the certificate is valid for 5 years and can be renewed upon request by the training centre (or network), centres will have to demonstrate that the recommendations made have been taken into account, and this will keep track of training quality even more. It also provides an occasion for trainees to share their own ideas on clinical education.

Next steps

Network of accredited centres

It is the intention of the Accreditation Committee to develop a network of accredited training centres, thus promoting interaction and collaboration between accredited training centres in areas linked to education and research.

European accreditation of training centres in paediatric respiratory medicine

The Accreditation of Training Centre in Paediatric Respiratory Medicine Working Group began drafting accreditation criteria relevant to Paediatric Respiratory Medicine in early 2015.

These criteria went through a validation process with a larger group of experts through an extensive survey in summer 2015 and will be revised accordingly by the working group in early 2016.

Publication of the accreditation criteria for Paediatric Respiratory Medicine is expected in June 2016.

Conclusion

Although available research correlating training accreditation with quality of patient care is scarce, the limited data available suggest that training accreditation positively influences trainees’ performance and overall satisfaction level. Through the accreditation process, targeted guidance is provided to training centres on specific measures to improve their educational and assessment methods.

It is the ERS/EBAP Accreditation Committee’s intention to continue developing the accreditation of the training centre programme, to incentivise training centres to improve their processes and to collaborate with other accredited training centres on the development and improvement of educational resources. This may call for the development of guidelines and further accreditation criteria specific to other respiratory medicine subspecialties.