Accreditation the Education Development Centers of Medical-Sciences Universities: Another Step toward Quality Improvement in Education.

BACKGROUND: In order to improve the quality of education in universities of medical sciences (UMS), and because of the key role of education development centers (EDCs), an accreditation scheme was developed to evaluate their performance.
METHOD: A group of experts in the medical education field was selected based on pre-defined criteria by EDC of Ministry of Health and Medical education. The team, worked intensively for 6 months to develop a list of essential standards to assess the performance of EDCs. Having checked for the content validity of standards, clear and measurable indicators were created via consensus. Then, required information were collected from UMS EDCs; the first round of accreditation was carried out just to check the acceptability of this scheme, and make force universities to prepare themselves for the next factual round of accreditation.
RESULTS: Five standards domains were developed as the conceptual framework for defining main categories of indicators. This included: governing and leadership, educational planning, faculty development, assessment and examination and research in education. Nearly all of UMS filled all required data forms precisely with minimum confusion which shows the practicality of this accreditation scheme.
CONCLUSION: It seems that the UMS have enough interest to provide required information for this accreditation scheme. However, in order to receive promising results, most of universities have to work intensively in order to prepare minimum levels in all required standards. However, it seems that in long term, implementation of a valid accreditation scheme plays an important role in improvement of the quality of medical education around the country.

Introduction

Capable staff and an efficient system are essential requirements in order to provide an acceptable level of health services in a community. Therefore, there are special quality assurance schemes in place to assess the excellence of education in medical sciences fields in most of developed countries. For such an assessment, accreditation is a high-quality evaluation scheme (1).

According to the definition introduced by the Council on Higher Education Accreditation (CHEA) in USA, accreditation is a process based on self and elite’s assessment for guaranteeing and improving the quality and responsibility of an institute or university course. In such a process, it can be specified whether the studied institute or program is based on standards issued by the CHEA and whether their performances fulfills the specified goals of the CHEA or not (2).

Thus, one of the most important advantages of accreditation is to ensure the government, society, learners, and the executive authorities of educational institutes about education quality and the quality of learners through guaranteeing the quality of the under-assessment unit. However, it should be considered that the value of accreditation is not limited to investigations and supervisions, but experiences of accreditation systems rather show that the activities of such systems will lead to establishment and reinforcement of internal assessment processes in educational institutes. In fact, internal and external assessment inside an accreditation structure can help such structures to make best use of the benefits of the both methods (3, 4).

EDCs were established in universities of medical sciences in Iran in the late 90s; their main aim has been to improve the education quality (5). The tasks for these centers, classified in five main fields: 1) educational planning 2) teacher training 3) research in education, 4) standardization of exams and teaching assessments, and 5) continuous medical education (6). However, by extension of the education fields in clinical and non-clinical fields, the tasks of EDC especially in recent years has been widening, which are as follows (7):

Directing, coordinating, and supervising the educational programs, the evaluation of new assessment/examination techniques, analyzing the results of exams, and comprehensive assessment of academic staff

Organizing and supervising activities in related to top and “gifted and talented” students

Supporting and scaling up researches in education

Coordinating activities in tele-education and e-learning, educational excellence

Coordinating and supervising the education development offices in schools and training hospitals

Coordinating all activities in related to academic staff development

Regarding the goal of these centers, it is important to make sure that EDCs are working efficiently. Therefore, formal monitoring of their performance is one of the crucial steps for constant improvement in medical education (8).

It is obvious that the activity and tasks of EDCs in all universities are not exactly the same. Therefore, creating appropriate and comprehensive indicators is a very critical step for any type of assessments of EDCs performance.

Based on the above logic, in the 42nd session of the council of education deputy managers on 17/12/2008 approved that only universities with standard EDCs can apply to establish new educational program (7). Therefore, a real need for accreditation of EDCs was materialized.

However, generating a valid tool with comprehensive standards is literally complicated. In order to address to this request, a process was defined in the education deputy of Ministry of health and medical education (MOHME) to designs an accurate model for accrediting the EDCs of UMS. In this paper, the steps and the process of this development is presented.

Methods

Firstly, the EDC of the MOHME formed a team of 9 experts. Special criteria were used to select the best candidates out of experts in the medical education field from the whole country. All of the team members had deep and vast knowledge in the medical education field, long term of experience as a manager in UMS, and were familiar with accreditation concepts.

The team initially decided to regulate their scheme based on Vertern, et al. model but with local adjustment. Accreditation elements in Vertern model (9) are as follows:

Explanation of pre-defined standards

Internal-assessment by institute

Making a team for external assessment

Site visit

Reporting the external team’s report

Assessment of the report by expert committee

Finalizing the report and decision making accordingly

Having reviewed all available documents and shared experiences, the team decided to design a set of standards in two levels which were basic (essential) and quality improvement standards. The basic standard means the required standard (the Musts) which each center has to prove complying with for accreditation. Quality improvement standard means the preferred standard (the Shoulds), specifying the development path of the center. Of course, only the basic standards were used in executing the presented model.

In order to maximize the content validity of these standards, the team regulated a conceptual framework of all activity in EDCs. Then, minimums and ideals of each EDC were defined through reviewing existed rules and regulations.

In the next step, team members wrote standards individually. Then, the written standards were criticized in the group and a final version of standards was developed by consensus.

After that, the team defined measurable indictors to assess standards. Each standard was explicitly explored the different sessions before finalizing indicators.

In the next phase, a clear guideline was writing which helped EDCs to understand the standards and indicators. In addition, the method of analysis of findings was clearly stated.

The five selected areas 1)governing and leadership, 2)educational planning, 3)faculty development, 4)assessment and examination, and 5)research in education are presented in Tables 1 along with their standards.

Table 1:

Basic standards of accreditation in EDC

Area	Standard	Standard Explanation
Governing & Leadership	A) strategic plan	The center ought to have a ratified strategic plan (i.e. a ratified plan confirmed and formally announced by high managers of the university) and shows that the plan is in line with the strategic plan of the university, executing its operational plans accordingly.
	B) interaction with other committees	The center ought to have appropriate and effectively interaction with educational and research committees within the university (like University Educational Council, University Research Council and subsidiary colleges).
	C) management	The center ought to have an independent manager (without any other key positions simultaneously) with adequate physical presence in the work place.
	D) manpower	The center ought to have manpower (e.g. faculty members, technical team, the committees’ members, and part-time or voluntary faculty members) proportional to its structure in order to execute the tasked issues.
	E) resources	The center has to have required equipment to access updated information in medical-sciences education field.
	F) website	The center ought to have an exclusive, dynamic, and updated website.
	G) structure	The center ought to have at least the units for education planning, assessment, faculty development, research in education and continuous training as its main units and should interact in planning with clinical skill labs
	H) physical space & equipment	The center ought to have a physical space including buildings and equipment proportional to its structure.
	I) educational scholarship	The center ought to prove it has appropriate facilities in order to receive, study, assess, and evaluate faculty members’ scholar activities
Education planning	A) curriculum	The center ought to have specific strategy for technical guidance and support of educational groups and colleges in planning, revising, and assessment of its curriculums
	B) training labs for clinical skills	The center ought to prove it can supervise the curriculums of clinical-skill labs
	C) responsive training	The center ought to form responsive training council using all related units in university for policy-making, planning, and supervising the desirable execution of responsive trainings over the university. Head of the council should be the university president confirmation.
	D) database of gifted and talented students	The university ought to have database of gifted and talented students
faculty development	A) assessment plan	The center ought to have a compiled educational assessment and should design the faculty members’ development plans on that basis.
	B) variety of curriculums	Growth and development plans of the center ought to be applicable and executable for different ranks of the board (newcomers, semi-experienced, and highly-experienced) with various forms and subjects including all roles of the board members.
	C) participation in the plans	The center ought to have appropriate strategies for the board members to participate in growth and development plans.
	D) effectiveness of the plans	The center ought to have a specific plan for assessment of its output and results of its plans.
Assessment and examination	A) modern assessment methods	The center ought to have a compiled plan to train the board members in new methods of assessing students as well as to execute them.
	B) analysis of the tests	The center ought to have an appropriate strategy to analyze multiple-choice tests and to deliver their feedbacks to the professors.
	C) scientific board members’ assessment	The center ought to have an appropriate strategy to assess the board members regularly and permanently at least from the viewpoint of the students and the feedback referred to the board members and the tested department head.
	D) internal assessment	The center ought to cooperate with educational groups in internal assessment area.
Research in Education	A) workflow of the plans	The center ought to have appropriate strategies to receive and specify the priority, methodological study, cost assessment, final ratification, and to send them to make contracts in educational plans.
	B) plans about research in education	The center ought to have appropriate strategies to specify priorities of research in education and should prove that priority specifications are based on educational problem-solving methods.
	C) publication of research results in education	The center ought to have an appropriate strategy to publish and present the results of its activities.

Data collection

A questioner was sent to UMS including operational definitions of each standard as the first step. UMS identified their situations through an internal assessment. The main goal of this step was to assess if UMS could find standards appropriate and applicable.

Then, the filled questioners were collected and assessed in the EDC of MOHME. A new national committee was formed from among those who had appropriate scientific and experimental experiences and did not hold any executive positions in educational management of the university. This team assessed all filled questioners and scored responses based on the guideline. In addition, they verified responses of UMS using independent sources.

Using cluster analysis, we assessed the similarities between the obtained scores for indicators; then we classified indictors into clusters in a way to minimize the distance of scores for ever indictor from the centers of clusters. SPSS version 11.5 was used of this analysis.

Results

Findings delineated that all medical-sciences EDCs of Iran were capable of taking part in the project and presented required data and information to execute the project which showed that the defined indices were measurable and perceivable.

As shown in Table 2, EDCs failed to pass in the following standards:

Table 2:

Result of basic standards of accreditation in medical-sciences EDCs of Iran

Field	Standard	Acceptable centers
		n	%
Governing & Leadership	A) strategic plan	39	97
	B) interaction with other institutes	36	83.72
	C) management	41	95.35
	D) manpower	30	69.77
	E) resources	26	60.47
	F) web site	40	93.02
	G) structure	32	74.42
	H) physical space & equipment	36	83.72
	I) educational scholarship	39	90.7
Education planning	A) curriculum	33	76.74
	B) training labs for clinical skills	42	97.67
	C) responsive training	21	48.84
	D) database of gifted and talented students	39	90.7
faculty development	A) assessment plan	34	79.7
	B) variety of curriculums	26	60.47
	C) participation in the plans	43	100
	D) effectiveness of the plans	32	74.42
Assessment	A) modern assessment methods	29	67.44
	B) analysis of the tests	33	76.74
	C) scientific board members’ assessment	40	93.2
	D) internal assessment	40	93.2
Research in Education	A) workflow of the plans	35	81.4
	B) plans about research in education	37	86.5
	C) publication of research results in education	36	83.72

- responsive training in education planning area

- resources in governing and leadership area

- variety of curriculums in the area of faculty members’ development

- modern assessment methods in the assessment area

On the other hand, the standards in the area of faculty members’ participation in projects (management, website, clinical skills training, assessment of faculty members, and internal assessment) had the best optimum levels.

The above case showed that EDCs in UMS have to pay more attention in the following areas in order to improve their performance:

- responsive training

- variety of curriculums,

- empowerment of faculty members

- applying modern assessment methods

Fixing 2 as the number of clusters, the averages of distances from the centers were dropped from 2.03 to 0.44 in cluster 1, and from 0.97 to 0.14 in cluster 2. Fixing 3 as the number of clusters, the averages of distances from centers were dropped from 1.78 to 0.68 in cluster 1, from 1.6 to 0.55 in cluster 2, and 2.19 to 0.17 in cluster 3.

Both methods of clustering dropped the dissimilarities within clusters significantly (P<0.001). However, the indicators within each cluster were conceptually different. For example, defining two clusters, the indicators in cluster 1 were 1) responsive training in educational planning, 2) the variety curriculums, 3) modern assessment methods, 4) manpower, 5) recourses, and 6) the structure the EDC.

Based on the above findings, it seems that although cluster analysis might find a few factors by combining indicators, the components of these created factors belongs to the indicators from every groups of governing and leadership, educational planning, faculty development, assessment and examination and research in education. In other words, the conceptual framework for defining main categories of indicators was not supported by the observed similarities among indicators within each category.

Discussion

Our findings showed that the generated indicators have enough comprehensiveness to cover nearly all activities within EDC. Moreover, universities did not have substantial difficulty to address these indicators. However, the pattern of responses and scores did not support the initial conceptual framework. It seems for development of a valid and feasible accreditation scheme of EDCs more experience and practice is needed. However the generated tools are an acceptable instrumentation to start with.

In recent decades, accreditation has been quickly spreading outside its initial borders (i.e. North America). This spread, by itself, resulted in some modifications in basic concepts of accreditation like the role of government as well as it’s voluntarily nature in accreditation. Accreditation which was initially an association to empower an organization, gradually changed into a tool in order to supervise and improve quality. So, in the definitions of accreditation among references outside the US, there is not any point regarding the non-governmental and voluntary concepts of accreditation (10). In USA, too, supervising the accreditation process is performed by the government; moreover, accrediting the accreditation institutes of medical education is also done by the US government (11). Methods and contents of modern accreditation systems are generally very similar to the previous and initial structures (10).

In Iran, from the 3rd national development plan from one decade ago, special attention has been directed to launch different sorts of assessments and accreditation systems to assess and improve the efficiency of educational institutes. Different enactments have numerously emphasized the importance of accreditation in medical universities and colleges (12), and among EDCs have a critical role to reform in medical education (13) for improving social accountability of medical education in Iran (14).

The generated indicators were based on a consensus among a group of experts in medical education in Iran. These experts defined indictors and their standards based on the current list of EDC tasks after a long discussion. Most of universities could address to these indicators easily without any objections; which might imply that the selected indicators and their standards were clear and comprehensive to cover almost all of their activities.

However, the results of factorial analysis were not compatible with the initial designed conceptual framework. It means that the strength of the association among EDC activities did not have a strong relationship with their labels and groups; as an example, “responsive training” and “modern assessment methods” had strong association although they are categorizing in two different groups. These discrepancies are very important and more targeted researches are recommended to address why some of a part activities have strong correlations, but more similar and conceptually linked activities did not strong correlations.

Although, this study might be unique, because of its approach to the concept of accreditation in the assessment of the quality EDCs activities; we did not check the validity of universities responses in field observation. Nonetheless, usually the responses of university in similar assessments were acceptable in national programs. However, for a real accreditation, it is recommended the main steps of the Vertern model to be followed (9).

Conclusion

The implementation of this accreditation scheme of the EDCs would be a critical point for quality improvement of medical education. However, based on our observation, UMS have to work intensively to fill their gaps to obtain basic standards in all indicators; otherwise, in the next round of accreditation a considerable number of EDCs will be disapproved.

Ethical considerations

Ethical issues (Including plagiarism, Informed Consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc) have been completely observed by the authors.