Designing Excellence and Quality Model for Training Centers of Primary Health Care: A Delphi Method Study.

BACKGROUND: Excellence and quality models are comprehensive methods for improving the quality of healthcare. The aim of this study was to design excellence and quality model for training centers of primary health care using Delphi method.
METHODS: In this study, Delphi method was used. First, comprehensive information were collected using literature review. In extracted references, 39 models were identified from 34 countries and related sub-criteria and standards were extracted from 34 models (from primary 39 models). Then primary pattern including 8 criteria, 55 sub-criteria, and 236 standards was developed as a Delphi questionnaire and evaluated in four stages by 9 specialists of health care system in Tabriz and 50 specialists from all around the country.
RESULTS: Designed primary model (8 criteria, 55 sub-criteria, and 236 standards) were concluded with 8 criteria, 45 sub-criteria, and 192 standards after 4 stages of evaluations by specialists. Major criteria of the model are leadership, strategic and operational planning, resource management, information analysis, human resources management, process management, costumer results, and functional results, where the top score was assigned as 1000 by specialists. Functional results had the maximum score of 195 whereas planning had the minimum score of 60. Furthermore the most and the least sub-criteria was for leadership with 10 sub-criteria and strategic planning with 3 sub-criteria, respectively.
CONCLUSION: The model that introduced in this research has been designed following 34 reference models of the world. This model could provide a proper frame for managers of health system in improving quality.

Introduction

Nowadays any organization necessarily needs auditing system in order to be informed about desirability and quality of its performances, especially in complex and dynamic environments (1–3). Many patterns and models have been used during formation of organizations in various periods (2, 4–6). In the last decade, models called excellence and quality gained many fans. Comparing with other models, usage of these models grew rapidly and contained helpful results. The main reason for this is due to their lack of need for designing and establishing complex systems for measuring performance and high flexibility of these models (considering rapid changes in internal and external conditions of organization). In these models, a set of criteria for evaluating organization and guidelines for evaluating have been provided (7–12). Organizational quality and excellence models are instruments for helping organizations in order to measure the degree of being in the way of organizational excellence and balanced growth. These models help organizations to identify differences by comparing their current and ideal status and then determine and perform solutions to optimize the current situation based on these differences and reason of their occurrence. The most popular of these models are The European Foundation for Quality Management (EFQM) excellence model, Malcolm Baldrige performance excellence model and Edwards Deming excellence model (13–15).

One of the most important of these organizations, are educational institutes evaluating the quality of which have become a major matter of concern for educational systems in most of the countries during the latest decades. In order to deal with this challenge, validation systems and excellence models have been formed in most of countries. However, there has been a little amount of effort to evaluate quality of educational institutes. These efforts would lead to improvement in quality if they are designed in the format of an evaluating system and are structured to implement it (16). Educational centers of primary health care are of educational centers in scope of activities of Health Department of Medical Universities which are required to train primary care providers (17). Considering the importance and significant role of primary care’s educational centers in providing proper human resources needed for health and medical system and finally undeniable role of these centers in making ground for providing health and medical services, the issue of evaluating and assuring quality of education in these centers is important and necessary.

According to the point that there are no proper tools available in Iran to evaluate and assure the quality of educational centers and there is not any official tools or checklist certified by the Ministry of Health and Medical Care in particular, the aim of this study was to design the model of excellence and quality for educational centers of primary care in Iran, using Delphi methods.

Materials and Methods

In this study, Delphi technique was used to design the national model of excellence and quality for educational centers of primary care (18–21). Sampling in this study is a purposed sampling method. In this method samples with the most and richest information are selected and sampling goes on until reaches informational saturation i.e. the point where new information could not be found (22–25). Finally, this study was conducted by attendance of 61 people including managers of training centers of primary health care, officials of the Health Center of East Azerbaijan province, Health Ministry’s authorities of educating health and medical primary care, universities’ authorities of training centers of primary health care, and nationwide professors of Department of Health and Medical Services. Inclusion and exclusion criteria included having Bachelor degree at least, having minimum of 5 years’ experience in the field of educating primary cares (managers of centers, preferably), having interest and will for attending the study.

First, a wide review of references such as internet, magazines websites, other verified websites, library websites, and databases of Medline, PubMed, and science direct was conducted in this study for a comprehensive investigate on quality models, excellence models, and accreditation models in education using keywords of quality models, excellence models, accreditation, primary health care, and education in the world. The time for collecting information was 1985 to 2014 and collected articles were chronically ordered in a way that newly published entries were analyzed first. The result of mentioned search was 643 document and entries and after checking the title and other parts, 427 of them were excluded due to weak relevance with subject and lack of mention about the standards of quality and excellence models. The result of study and evaluation of 216 articles, were 39 models in accordance with educational and medical system, economic and cultural conditions, and models expected by the Ministry of Health and Medical Education from 34 countries.

In this stage, criterion and related scores were extracted in a table and then criterion of the primary model based on criterion with maximum frequency (based on the results of models and ideas of team members, 8 criterion were selected) and in the next level with investigating models and selected criterion, proper sub-criterion and standards were provided.

In the stage after collecting information and study on 39 models in the world, criteria, sub-criteria, and standards of primary models were extracted based on available information and studied models and the primary model was designed based on studied models and suggestions of research team. After designing primary model including 8 criteria, 55 sub-criteria, and 236 standards, it was designed in the form of Delphi questionnaire including two parts of “importance” and “applicability”, each containing 5 Likert scales (appendix1) and was distributed by Delphi method among 11 specialists and experts of health and medical system of East Azerbaijan province, and finally 2 persons quitted helping model due to being busy, and 9 questionnaires were completed and sent back.

Then the mean of persons’ selections was determined and standards with a mean of less than 3 were omitted, standards with the mean of 4 entered to the second Delphi phase and standards with mean of 5 were definitely accepted. After second Delphi phase in provincial level, Focus Group Discussion (FGD) meeting was held with attendance of all 9 persons and the model underwent correction and final completion in provincial level. The primary model was finalized in provincial level with 8 criteria, 45 sub-criteria, and 210 standards, after a 5-hour meeting. After completing and correcting the model in provincial level, corrective questionnaire was sent to 50 authorities of training center of primary health care in the country, among whom 33 persons answered and sent them back. Again the mean of persons’ selections was determined and standards below 3 were omitted, standards with score of 4 entered to the second country Delphi phase, and standards with score of 5 were finalized. In this stage the questionnaire decreased to 8 criteria, 45 sub-criteria, and 197 standards. In this phase of Delphi due to high number of standards with mean of 4, Delphi technique was performed for the second time in country level. In this level questionnaire was sent to 10 authorities of training center of primary health care in the country and after receiving questionnaires and analyzing them- like previous levels- the number of criteria, sub-criteria, and standards changed to 8, 45, and 192 respectively and the designed model was finalized with these numbers.

After finalizing model, for investigating it, using Nominal Group method with presence of study team members, authority of Tabriz Training of Primary Care, and two faculty members, tool and metrics for evaluating model were provided which included 611 metrics guides and standards (appendix 2). One of centers for training primary care in province was selected for conducting evaluation (Alzahra (PBUH)), first necessary coordination was arranged with related authorities and standards were provided to authority of primary care training center of (Alzahra (PBUH)) one week before accreditation. Then research team referred to mentioned training center and started to complete the checklist with cooperation of authorities and trainers. Checklist completion was done through interview with authorities and trainers of center, investigating documents, and running necessary observations. Based on the results of evaluation, some changes occurred in standards, evaluation method, and scores of standards.

Ethical issues of this research included acquiring verbal consent from participants for participating in this study, keeping privacy of people’s information, insuring participants about not releasing information other than objectives, having abandoned right in any level of research, and getting permission if needed.

Results

After collecting information and studying the models, criteria, sub-criteria, and standards, primary model was designed. Leadership with 10 sub-criteria and strategic planning with 3 sub-criteria, respectively had the most and least sub-criteria and also process management with 46 and planning with 14 standards included the most and least standards respectively. Functional results with score of 190 and planning with score of 60 were the maximum and minimum scores. Table 1 summarizes this information.

Table 1:

Criteria and related scores in the primary model

Criteria	Total frequency	Scores in %	Score	Number of standards	Number of sub-criteria
Leadership	160	16	160	45	12
Planning	220	6	60	14	3
Information management	290	7	70	17	5
Process management	470	18	180	46	11
Resource management	550	8	80	21	4
Customer management	660	11	110	29	5
Staff management	810	15	150	35	8
Functional results	1000	19	190	29	7
	1000	100	1000	236	55

Standards with scores of “importance” about 90% and “applicability” about 80% from experts’ point of view were finalized and those with scores of importance and applicability under 60% were omitted. Total score of standards is 1000 in which leadership with 10 sub-criteria and strategic and operational planning with 3 sub-criteria have the most and least sub-criteria respectively. In addition, process management with 38 standards and strategic and operational planning with 11 standards have the most and least standards.

After analyzing the results of the country Delphi survey phase, Delphi technique was implemented since there were a high number of standards with score of 4. Results of two Delphi phases have been shown in Table 2 .

Table 2:

Finalized model after two phases of country evaluation

Criteria	Cumulative frequency	Score in %	Score	Number of standards	Number of sub-criteria
leadership	150	15	150	35	10
Strategic and operational planning	210	6	60	9	3
Information analysis	300	9	90	12	4
Process management	485	18/5	185	33	8
Resource and facility management	555	7	70	14	3
Customer management	685	13	130	24	4
Human resources management	805	12	120	28	6
Functional key results	1000	19/5	195	37	7
	1000	100	1000	192	45

After final Delphi phases, excellence and quality model for training centers of primary health care was finalized with 8 criteria, 45 sub-criteria, and 192 standards. Functional results with 37 standards and strategic and operational planning with 9 standards had the most and least standards.

For any of criteria a definition, which has been mentioned in most of the resources, has been chosen in order to prevent ambiguities and a common perspective of criteria would be provided for all users (appendix 3).

In the next level, after finalizing model, for each standard a measurement guide (declaring subjective) and factors measuring standard (measurement components) were provided. For example, in leadership criterion the first sub-criterion was creating organizational values, goal, mission, and culture and its first standard was “revision and development of organizational mission with cooperation of authorities, trainers, and other staff”. Scores obtained by Training Primary Care center of Alzahra (PBUH) are shown in Fig. 1 distinctively for eight criteria of the model.

Fig. 1:

Results of evaluating Tabriz Training Primary Care center of Alzahra

As it is seen in Fig. 1, primary care training center has gained the highest score in field of human resources’ management and the lowest score in field of analyzing information.

Discussion

According to importance and necessity of paying attention to quality in educational centers and also paying attention to the main role of quality and excellence models in improvement of quality of educational centers, special attention should be paid to the subject of quality and excellence models in educational centers especially in health and medical system. Today the need for designing and executing quality and excellence models in educational system of Iran is felt more than any other time. Among different educational centers in the country, educational centers of primary care are very important and need serious attention for educational development through taking advantages of methods for improving quality and qualitative evaluation, since they train staff who are in the first line of providing medical services to people, have the widest interaction with them, and have low education. The designed model in this study (8 criteria, 55 sub-criteria, and 236 standards) was finalized as a model with 8 criteria, 45 sub-criteria, and 192 standards after four phases of experts’ evaluation.

Iranian Education Quality Management model has been developed with major objective of improving quality of educational services in universities under authority of the Ministry of Health and Medical education and has 7 criteria (leadership, strategic planning, focus on students, beneficiaries and market, measurement, knowledge analysis and management, focus on manpower, process and result management). It has one criterion less than our model’s criteria and is about resource management. Considering great deal of resources available for universities of Medical Sciences and their importance, it seems that it is better to add this item to the model above. This model has 37 sub-criteria, which has 13 items less than sub-criteria of our model. In addition, this model has 224 standards, having 32 more standards comparing with our model with 192 standards. Total score of this model is 1000 as well as that of our model (26).

Malcolm Baldrige model (model of excellence in education) has 7 major criteria that is one criterion less than that of our model (human resources management). It also includes 18 sub-criteria, which has 27 sub-criteria less than that of our model. Total score of this model is 1000 as much as that of ours (27, 28). EFQM model (model of quality reward in higher education) has 9 major branches which is one branch or criterion more than that of our model for excellence and quality of primary cares educational centers and is for society results. The reason of this is due to relatively low communication of primary cares training centers with society. In addition, this model has 32 sub-criteria, which have 13 sub-criteria less than that of our model. Total score of this model is 1000 as much as that of our model (12, 29)

In this study, accreditation models were used alongside with quality and excellence models in order to gain higher richness in standards and a more complete model. Reviewing the texts shows that accreditation also has been neglected in Iran and this is obviously seen in educational sector, especially in primary care education part. Among few studies conducted in this field, the study of Naseri et al. (30) on designing accreditation standards of clinical nursing education using international standards is notable. This led to development of 55 standards in 5 domains and standards with 90% acceptability were finalized which is relatively in accordance with the results of current study. In this study, researchers linked the lack of acceptance of some standards to the nursing educational system in Iran which is the same case in this study in applicability aspect, in a way that most of reporters claim that despite high importance of some standards they have limited applicability due to the bureaucratic and cultural background of the country. In other study by Pazargadi et al. (31) aiming to provide a pattern for accreditation of universities of medical sciences, developed a model with 216 functional indicators in format of 16 major components with acceptance level of about 70%. In order to do so, researchers suggested a four-stage process of asking for accreditation, determining standards, self-evaluation, and peer evaluation for performing effective accreditation. In this study, researchers divided their designed model into two parts of major components and functional indicators, while in our study the model has been formed of three parts of criteria, sub-criteria, and standards. In terms of number, this study has 216 indicators, which are relatively similar to 192 standards of our study. About acceptance of standards from experts’ points of view, acceptance level of 70% in this study is relatively low in comparison with 80% and 90% of our study. The Organization of Technical and Professional Training of the country has divided accreditation standards into three parts of resource, process, and functional standards including 59 standards in the format of 10 axes in accreditation regulation of open technical and professional institute, which in this view has a great difference with designed model of the current study (32).

In this study, the quality and excellence model for primary health care training center has been designed using 39 famous and verified models in the world and it seems that has a high degree of comprehensiveness. Therefore, it may be applicable in other educational centers of the country with a few changes. This study has also some disadvantages including lack of similar centers in abroad, and lack of foreign experiences in this field where there were no standards for measuring primary cares educational centers. For this reason, we had to use similar standards from other educational centers and modify those standards using ideas of 60 national experts and specialist in a way that a proper model for primary cares educational centers was designed.

Under study, educational center gained the lowest score in information analysis criterion. This score is equal to 35% of total score in this field. This shows the weak performance of center in field of information analysis. One of the reasons to this issue could be center’s neglect about analysis of collected information. Since information is the basis of decision-making process in all organizations, ignoring the use of collected information analysis and interpretation could lead to unscientific decision-makings.

In addition, due to time and financial limitations, it was not possible to travel to all provinces and take advantages of ideas and experiments of all experts but it was tried to use ideas of nearly all experts responsible for primary care education in universities and ministry of health and medical education. Therefore, it is possible to claim that the current model is very rich in terms of using experts’ opinions. Another limitation of this study is lack of on time sending of standards to the authorities of primary care training center. It is suggested that in order to gain better results standards should be sent to centers at least 6 month before evaluation.

Conclusion

The model designed in this study could be completed and developed following further studies and acquiring specialized suggestions. This model needs pilot execution in some training centers of different provinces so if it was fruitful, it could be implemented and executed as a national program. During pilot study and afterward, finalized model should continuously be evaluated and in order to develop model, revisions should be considered based on identified weaknesses. In addition, as it was mentioned before, current model could be a base to design specific model of quality and excellence models for other educational centers of the country which needs effort of related experts and full support of ministry of health and medical educations.

Ethical considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.

Criterion :leadership
Sub-criterion: creating values, vision, mission, and organizational culture
Standard: Review and development of organizational mission with cooperation of authorities, trainers, health workers, and other staff
Your comment:
Applicability					Importance
5	4	3	2	1	5	4	3	2	1

Criterion :leadership number of sub-criterion: 10	number of standards: 35		total score:150		gained score
Sub-criterion 1:creating values, vision, mission, and organizational culture number of standards:5 total score:10 gained score:
Standard 1 :Review and development of organizational mission with cooperation of authorities, trainers, health workers, and other staff
Number of metrics: 2	Score: 2	Gained score
Measures	Score of measure	Yes= complete score	Somehow/sometimes =50% of score (changing depending on evaluator’s idea)	No= without score or zero	Documents	Considerations
A) Whether meetings are held to review and develop organizational mission with attendance of authorities, trainers, health workers and other staff?	1				Proceedings
B) Whether feedback is asked from authorities, trainers, health workers and other staff about organizational mission or development and review of it?	1				Interview

Criterion	Definition
Leadership	Assurance about the issue that how organization will involve its staff, managers, and all organizational levels in continuous improvement of quality and excellent performance, in order to reach its goals.
Functional key results	Evaluating performance of organization in two domains: financial performance, and executive and educational performance
Process management	Assurance about design, management, evaluation, and improvement of key processes for reaching excellent and outstanding results
Human resources management	Evaluating organizational plans and developing manpower for taking maximum benefits of manpower
Customer management	Evaluating organization’s ability in identifying needs and expectations of customers for gaining their satisfaction and approval and ability of organization for developing relationships with customers
Resources and facilities management	Evaluating status of different organizational resources management including: financial, materials and facilities, technology, intellectual capitals, estates and physical spaces
Strategic and operational planning	Evaluating organizational development, communications, execution and improvement of strategies and policies of organization to gain a perfect performance and better situation
Information analysis	Evaluating and using information and informational within and outside the organization