Surveyor Management of Hospital Accreditation Program: A Thematic Analysis Conducted in Iran.

BACKGROUND: The surveyors in hospital accreditation program are considered as the core of accreditation programs. So, the reliability and validity of the accreditation program heavily depend on their performance.
OBJECTIVES: This study aimed to identify the dimensions and factors affecting surveyor management of hospital accreditation programs in Iran.
MATERIALS AND METHODS: This qualitative study used a thematic analysis method, and was performed in Iran in 2014. The study participants included experts in the field of hospital accreditation, and were derived from three groups: 1. Policy-makers, administrators, and surveyors of the accreditation bureau, the ministry of health and medical education, Iranian universities of medical science; 2. Healthcare service providers, and 3. University professors and faculty members. The data were collected using semi-structured in-depth interviews. Following text transcription and control of compliance with the original text, MAXQDA10 software was used to code, classify, and organize the interviews in six stages.
RESULTS: The findings from the analysis of 21 interviews were first classified in the form of 1347 semantic units, 11 themes, 17 sub-themes, and 248 codes. These were further discussed by an expert panel, which then resulted in the emergence of seven main themes - selection and recruitment of the surveyor team, organization of the surveyor team, planning to perform surveys, surveyor motivation and retention, surveyor training, surveyor assessment, and recommendations - as well as 27 sub-themes, and 112 codes.
CONCLUSIONS: The dimensions and variables affecting the surveyors' management were identified and classified on the basis of existing scientific methods in the form of a conceptual framework. Using the results of this study, it would certainly be possible to take a great step toward enhancing the reliability of surveys and the quality and safety of services, while effectively managing accreditation program surveyors.

1. Background

Healthcare organizations nowadays are faced with important challenges due to the competition for provision of services. Hence, not only healthcare policy-makers and managers, but also healthcare providers and patients find standardization, accreditation, and assessment of health services providers inevitable in order to improve the quality and safety of services (1-5).

In recent years, health systems authorities in various countries have employed a number of techniques to enhance the quality and safety of healthcare and optimize management, and these can be briefly examined in two main categories:

The models that, by performing quality-based extra-organization evaluations, oblige organizations to show a more serious commitment to enhancing the quality of their services and products.

The procedures that contribute to high-quality management in organizations. In this context, the accreditation of the first group and the clinical governance of the second group play particular roles in the health sector (6-11).

In the early 20th century, accreditation was used as a mechanism to improve the quality of clinical practice in hospitals. The accreditation program was officially formed in the USA in 1951, under the title “joint commission and accreditation for healthcare organization”. The program was then extended to Canada and Australia in the 1960s and 1970s, to Europe in 1980, and finally to the entire world in 1990 (12-16). Accreditation is generally a voluntary program presented by non-governmental organizations. Using this program, health service providers are assessed by externally trained surveyors, on the basis of compliance with pre-established standards (12, 16, 17). The accreditation programs in various countries are determined based on the characteristics and level of development of health services (12, 18). Overall, although several studies assessing hospital accreditation programs have been conducted worldwide over a number of years, only a limited number of investigations have been carried out to assess various aspects of the implementation of the program and the resulting effects. Nevertheless, 10 major mainstays of hospital accreditation programs have been studied in a systematic review (14):

Professions’ attitudes to accreditation

Promoting change

Organizational impact

Financial impact

Quality measures

Program assessment

Consumer views or patient satisfaction

Public disclosure

Professional development

Surveyor issues

With the exception of two of these mainstays, 2 and 9, the results were inconsistent. A lack of sufficient studies had been performed to evaluate three mainstays: 1, 8, and 10 (14, 19). However, accreditation program surveyors are considered as the core of accreditation programs and the validity of an accreditation program to a great extent depends on their performance (15, 20). Therefore, an accreditation program surveyor must be sufficiently proficient in the areas under assessment to be capable of utilizing standards in an accurate, flexible, and balanced manner. These surveyors should have a commitment to quality and innovation, with an acceptable level of techniques, competencies, and organizational knowledge (21). In general, reliability, consistency, and quality of assessments in an accreditation program are closely associated with the nature and quality of the issues, such as surveyor selection, training, support, and stimulation, which is the so-called surveyor management accreditation program, and it is highly important to take advantage of specified indicators for this purpose (21, 22). In Iran, the office of supervision and accreditation of medical affairs became affiliated with the ministry of health and medical education in 2010 to ensure that provision of high-quality, safe services replaced the old model of assessment, and employing the accreditation program as a systematic and purposeful tool to achieve this aim. The ministry of health notified medical universities across the country of the accreditation standards in January 2012 (9). However, a specific mechanism for the hospital accreditation of surveyor management was not considered, and surveyors were mostly selected and embedded into survey teams without implementation of surveyor management processes.

2. Objectives

This study aimed to identify aspects and components affecting accreditation program surveyor management in Iran, in order to create a specified pattern, and enable higher quality, more reliable, and more accurate surveys, and thus fill the research gap in this area.

3. Materials and Methods

This qualitative Iranian study was performed from June 2014 through September 2014 using a thematic analysis method. The participants consisted of experts in the area of hospital accreditation derived from three groups: 1. Policy-makers, administrators, and surveyors of the accreditation bureau, ministry of health and medical education, as well as universities of medical sciences; 2. Healthcare service providers; and 3. University professors. The inclusion criteria were at least 3 years of work experience in the field of hospital accreditation and activity as a surveyor in the surveys team. The data were collected using semi-structured, in-depth face-to-face, separate interviews. The participants were selected on the basis of purposeful criterion sampling, which continued until data were saturated. Thus, 21 individuals participated in the study. The main interview questions were modified after one stage of a pilot interview, and after receiving the opinions of four of the experts, and the participants of the three groups were provided with 10 questions (Box 1).

Box 1.

Some of the Interview Questions

Questions
1- In your opinion, which indicators should be considered in the recruitment and selection of hospital accreditation program surveyors?
2- In your opinion, what characteristics should a surveyor team have?
3- What is your opinion of the process, content, and method of training of surveyors?
4- In your opinion, which points should be considered to retain and motivate the surveyors?
5- In your opinion, which issues must be considered in order to enhance the inter-rater reliability of surveyor teams?

The study was conducted as a part of a PhD project and benefited from a small grant from the school of public health, Tehran University of Medical Sciences, Iran. The code of ethical approval was 8921532002/1. In accordance with ethical considerations, the interviews were recorded after explaining the purpose of the plan to participants and obtaining their consent. Each interview lasted 40 to 90 minutes and the total time spent interviewing was approximately 22 hours. The interview audio file was transcribed and controlled in compliance with the original audio file, and MAXQDA10 was then used to classify and organize the information in six phases (23), as follows: 1) familiarization with data, 2) conceptual framework identification and generation of initial codes, 3) searching for themes and primary themes formation, 4) themes revision, 5) themes definition and naming, and 6) reporting. The data coding process was performed by one of the researchers.

In order to confirm data trustworthiness and study credibility, peer-check and member-check methods were used. The credibility of data analysis and coding was obtained using of the agreed codes divided by assigned codes by two encoders (24). The credibility between the two different coders was 83%. To assess the transferability of the codes, which includes the relationship between major and minor codes and the sub-codes resolution, coding control by 10% of respondents was used and the rate of 89% was obtained. In this regard, the results of the data analysis and classification were confirmed by two experts.

4. Results

4.1. Demographic Finding

The average age of the participants was 43 years (SD: 6, Range: 30 - 55) and 72% were aged 40 to 50 years. A total of 21 experts in three groups were interviewed, of whom 62% were female, and 81% were in the first group. Of the respondents, 90% had over 10 years of professional experience with education levels in the fields of medicine (33%), nursing (33%), and health services management (24%). One of the participants in the first group, and two of the respondents in the third group had a PhD in healthcare management, with over 10 years of work experience regarding hospital evaluation and accreditation (Table 1).

Table 1.

Demographic Characteristics of the Study Participants[a,b]

	First Group	Second Group	Third Group	Total
Gender
Female	11 (84.6)	1 (1.1)	1 (1.1)	13 (62)
Male	6 (75)	1 (12.5)	1 (12.5)	8 (38)
Field of study
Physician	4 (57.1)	2 (28.6)	1 (14.3)	7 (33)
Nurse	7 (100)	-	-	7 (33)
Healthcare management	4 (80)	-	1 (20)	5 (24)
Other	2 (100)	-	-	2 (10)
Work experience
5 to 10 years	2 (100)	-	-	2 (10)
Over 10 years	15 (79)	2 (10.5)	2 (10.5)	19 (90)

aValues are presented as No. (%).

bTotal number of subjects is 21.

4.2. Coding and Interview Classification Findings

First, 1347 semantic units were classified in the form of 11 themes, 17 sub-themes, and 248 codes on the basis of a primary analysis. Finally, the data were classified into seven main themes, 27 sub-themes, and 112 codes using the opinions of experts in the field of evaluation, hospitals accreditation, and health care management, following some expert panel sessions (Box 2).

Box 2.

Themes and Sub-Themes of Surveyor Management in an Accreditation Program

Themes and Sub Themes
Recruitment and selection of survey team
Organizational and job commitment
Work experience
Level of education and field of study
Personality-behavioral characteristics
Surveyors staffing plan
Academic qualifications
Staffing procedures
Organizing the survey team
Surveyor team composition
Surveyor team leader
Number of team members
Factors affecting of the number of team members
Surveyors’ type of cooperation
Inter-rater reliability
Planning to carry out surveys
Planning for distribution and rotation of surveyors
Determining framework for the number and timing of survey
Actions before, during, and after the survey
Financing and payment
Surveyors’ motivation and retention
External-hygiene factors
Internal-motivational factors
Surveyors’ training
Policy-making, goal-setting, and educational needs
Training content
Training methodologies and assessment of training course
Surveyors’ assessment
Methods and types of assessment
Assessment area
Continuous and periodic assessment
Developing suggestions
Policy-making
Organizational support

4.3. Recruitment and Selection of Survey Team

From the perspective of the study participants that embraced most codes, one of the most influential factors in the surveyor management of the hospital accreditation program in Iran was definitely that of issues related to the recruitment and selection of surveyors for this program. They believed that recruitment and selection of an appropriate surveyor can not only enhance the credibility of the assessments, but also influence other surveyor management processes: “The selection of an appropriate surveyor makes the learning process short, effective, and economical (P: 8)”.

Some interviewees believed that the job of surveyor is highly important, since not only do their performance and the results of their work affect the validity and reliability of accreditation, but also it is they who primarily influence a hospital’s financial cycle. In the view of the study participants, the recruitment and selection of surveyors should be based on issues such as staffing plan, staffing procedures, personality-behavioural characteristics, academic qualifications, level of education and field of study, work experience, and organizational commitment (Box 3).

Box 3.

Sub-Themes and Codes Related to the Surveyor’s Selection and Recruitment Theme in the Accreditation

Sub Themes and Codes
Surveyors staffing plan
Select from employees of the ministry of health and medical education, universities of medical sciences and hospitals; select from retired employees of universities of medical sciences and ministry of health and medical education; select by employment calling; select through training courses
Having clear goals and expectations of the surveyors
Having scientific competencies
Having management knowledge
Having accreditation and assessment knowledge
Having clinical knowledge
Level and type of education
Paramedical courses
Medical field
Having a relevant qualification
Personality - behavioural characteristics
Personal, social, and professional ethics
Communication skills (EQ)
Control and decision-making
Organizational and job commitment
Belief and interest in accreditation
Commitment to goals
Work experience
Experience at managerial levels
Hospital and clinical experience

4.4. Organizing the Survey Team

The number of team members, factors affecting the number of team members, team composition, team leader, team members’ type of cooperation, and inter-rater reliability were among the issues raised regarding this theme. The study participants mostly believed that the number of team members for the Iran hospital accreditation program should not exceed 10, and ideally, and under the best conditions, it is better to reduce the number to three to five people. “The number of team members should not be high because it leads to reduced coordination between them, and hospitals may face difficulties, so I think three to five are sufficient for the surveyor team (P: 5)”.

“We have used 20 to 25 in our teams, but I disagree with this. More limited team numbers mean more homogeneity, less bias, and there is a greater possibility of bringing together ideas and methods (P: 12)”.

One of the most important issues in organizing the survey team was inter-rater reliability and strategies to increase this. For example, the interviewees stated that standardizing the number of team members, selecting the right people, surveyor style, independence and impartiality, equalization of ideas, workload and fatigue, and avoidance of personal judgments are among the factors affecting this variable. “Inter-rater reliability is an important issue because we observed many cases with the same standards, conditions, and the same document surveyors gave different scores (P:12)”.

“Now, as the number of teams’ members is high, but on the other hand, they are not experts, I think inter-rater reliability is low (P:21)”.

4.5. Planning to Carry Out Surveys

Financing procedures, surveyors’ payment amounts and mechanisms, paying the costs on assessments, distribution planning, surveyors’ shifts, and determining the framework for the number and timing of surveys were issues that were considered in this theme (Box 4). For example, the opinions regarding financing procedures, amount and payment mechanisms to surveyors indicated that there is no integrated and reasonable mechanism for surveyors’ payment in the current accreditation program. “These people are not well-paid and the payments are minor (P: 1)”.

Box 4.

Sub-Themes and Codes Related to Planning to Carry Out Surveys Theme in Surveyor Management of Accreditation Program

Sub Themes and Codes
Financing and payment
Amount and payment mechanisms to surveyors
Survey costs payment
Need to plan a budget for accreditation by universities
Planning for distribution and rotation of surveyors
Necessity for different hospitals’ assessment by surveyors
Proportionality between the surveyors and environment
Surveyors’ rotation
Determination of the framework for the number and timing of surveys
Four surveys conducted per month
Assign 2 to 7 days for each survey
Commensurate with the size and complexity of services
Commensurate with the size of the hospital

The study participants believed that a budgeting plan and a certain credit must be specified for conducting surveys. Their views indicated that the costs should be paid by the ministry of health and medical education and/or hospitals. “I think the assessment tasks, especially as a senior surveyor, are a ministerial job, and the ministry should consider a special credit in this regard. Also, part of the cost should be paid by hospitals to prevent them from indifference to some issues (P: 12)”.

4.6. Surveyors’ Motivation and Retention

In the experts’ view, financial incentives and payments, receiving certificates and licenses, career promotion as a senior surveyor or presence of a ranking mechanism, respect for status, considering opinions, permission to continue education, performance feedback, and holding assemblies of surveyors are among the issues that can be used as triggers to motivate and retain individuals as surveyors in the hospital accreditation program in Iran. These factors are classified into two categories, in accordance with Herzberg’s motivation-hygiene theory: external-hygienic and internal-motivational. In the participants’ view, financial incentives and salaries are generally considered the main motivators for surveyors. “I think the first motivator is financial. Not having financial problems is very important. Surveyors should not be in conditions in which they are induced to give high scores and leave aside fairness (P: 18)”.

4.7. Surveyors Training

Although accreditation Program surveyor training is critical, the opinions of the study participants indicate that, unfortunately, it did not receive much attention, and most surveyors did not have an integrated and purposeful training program, especially in the early stages of accreditation program implementation. “Surveyors were not well trained (P: 8)”.

“Current surveyors were neither well selected, nor properly trained (Participant: 10)”. In general, the content was categorized under three sub-themes: 1) educational goal-setting and policy-making (including goal-based education, in accordance with job description, training needs assessment, and provision of centralized and integrated training), 2) training content, and 3) training methodology and necessity to perform an assessment of the training course. According to the respondents, the provision of training in skills in practical methods, such as training in the field, was of greater efficacy. “The field education method is very good, but we cannot provide this number of surveyors with it (P: 2)”.

4.8. Surveyors’ Assessment

The comments on this theme indicated that surveyors’ assessments must include a continuous and periodic process in the area of knowledge, attitude, behavior, and performance. They believed that for conducting such assessments, it is better to utilize certain methods, including a cross-check assessment (assessment of surveyor performance carried out by professional groups, which is usually conducted after the primary assessment and specifies any deviation of assessment performance), designing a standard check list, and use of senior surveyors, self-assessment, and peer assessments. “We can take random samples of conducted assessments by surveyors’ teams and check their scores to see how they differed (P: 11)”.

4.9. Developing Suggestions

This theme was classified in response to recommendations regarding proposals to improve the surveyor management of the hospital accreditation program in Iran into two sub-themes of policy-making and organizational support. “I think political and organizational support should exist for the survey and surveyors (P: 11)”.

The main issues raised in this theme were the importance of addressing the surveyors’ management in the hospital accreditation program in Iran, the necessity for benchmarking from successful organizations in hospital accreditation, the necessity for cultural promotion regarding survey and surveyors, the necessity for an independent, non-governmental organization to perform a survey, the design of an integrated system of surveyors’ information management, and the necessity to pay attention to occupational hazards and burnout faced by surveyors. “We need a comprehensive information system and a database of surveyors (P: 19)”.

5. Discussion

The results relevant to the theme of surveyors’ recruitment and selection showed that the qualified surveyors assigned to a hospital accreditation program are those who are equipped with simultaneous clinical, administrative, and accreditation knowledge. Thus, it is known that taking advantage of clinical and hospital practice experiences is the main requirement for the entrance of surveyors into this area. The participants believed that hospital accreditation program surveyors must have certain personality-behavioral characteristics in the areas of personal-social and professional ethics. Some of the main personality-behavioral characteristics considered by the respondents included communication skills, flexibility, confidence, accountability, honesty, open-mindedness, and impartiality. Shaw showed that a surveyor should be selected on the basis of identification and definition of the components of employment, and taking advantage of a set of competencies and qualifications (25). Another study showed that surveyors should have both clinical and managerial experience (26). Many of the validity accreditor organizations also believe that surveyors should have appropriate, relevant, and long-term experience (27).

With regard to the theme of organizing the surveyor team, the findings indicated that such a team should be composed of graduates of medicine, nursing, and healthcare management. There should be a balance between clinical and managerial experience, and professionals from other disciplines must be used only when necessary, and only for providing advice not for scoring. According to the interviewees, the appropriate number of members for a hospital accreditation team in Iran is three to 10. However, this number can vary in proportion to the size of the hospital and the time allocated to the survey. Other studies investigating the composition of the assessment team noted that a balance between managerial and clinical experience should be established. These studies asserted that the number of members comprising the main core of survey teams should be three to five (15, 27).

The respondents indicated that inter-rater reliability within a group and its promotion strategies are issues of importance in the theme of organizing a surveyor team. For example, some stated that the standardization of the number of team members, selection of the right people, surveyor style, independence and impartiality, equalization of ideas through training, workload and fatigue, and avoidance of personal judgments are factors that can affect inter-rater reliability within a group. Frisino studied issues such as surveyor training, continuous assessment of surveyors, and their impartiality to increase the credibility of accreditor organizations. The current study also pointed to the fact that surveyors must be experienced and act in accordance with the established standards (28).

In the theme of planning to conduct a survey, determination of a time limit to perform surveys was among the issues affecting the surveyors’ performance and thus survey validity. The respondents believed that four surveys per month are suitable. Of course, the size and complexity of services and hospital size were factors that were influential in determining the number of surveys that should be conducted within a month. Other issues discussed in this theme were the role of surveyors as consultants and trainers; most of the respondents believed that, considering the opportunity created by accreditation program, it is better for trained surveyors to train hospitals, as well as people, in order to enhance service quality. Low indicated that surveyors should play the role of counselor and trainer for hospitals, to enable the latter to provide high-quality care and to comply with acceptable standards (15).

The results regarding surveyors’ motivation showed that factors such as good salaries, respect for surveyor dignity, and the opportunity to learn and participate in follow-up educational training sessions were the main motivators for surveyors. Greenfield et al. showed that the motivation of surveyors with regard to participating in the accreditation program was to contribute to improving quality and safety and to create an opportunity to increase capacities (29, 30). It should be noted that in the hospital accreditation program in Iran, surveyors are selected from personnel working in universities and also in the ministry of health and medical education. Each of these has specified tasks and salaries in their relevant organization, and unfortunately they are not paid well.

The participants of the present study considered surveyor training and relevant issues to be of great importance. They believed that training should be continual and purposeful, and in accordance with job description. Training courses should be based on a needs assessment, consisting of primary training and retraining. The time devoted to primary training should be lengthy, while the retraining should be limited. The respondents believed that suitable training would be field-based and carried out in a practical manner, in the form of apprenticeships and the holding of virtual assessment sessions, and finally results in equalization of surveyors. They believed that the content of education should cover scientific and technical issues related to standards, communication skills, interviewing, observing, reporting, and teamwork.

Previously conducted studies have shown that the training of a surveyor includes the provision of information regarding validity assessment organizations, the role of surveyor, standard interpretation, and assessments’ conformity with standards and techniques. These studies indicated the range of methods, such as workshops, teleconferences, self-study tasks, and holding mock surveyor training assessments (21, 27).

The theme of surveyor assessment of a hospital accreditation program in Iran comprises two core sub-themes, namely assessment area and types and methods. The respondents believed that assessments must be conducted continuously to identify deviations and also to identify the surveyors that require further .training They believed that the surveyors’ assessment must not only assess the surveyor’s knowledge, attitude, and practice, but it should also be capable of assessing their behavior. Therefore, the participants introduced two methods: cross-check assessment and assessment by a senior surveyor (using standard and integrated check lists).

Designing of the surveyor information system, lack of financial communication between the surveyor and hospitals, surveyor occupational fatigue, surveyor occupational risk, and provision of appropriate support services were among the issues introduced in the theme of developmental suggestions for the management of hospital accreditation programs. The respondents considered the presence of an information system relevant to surveyors as useful because they believed that this system gives access to surveyors and all the information provided, in addition to facilitating their management. The findings have also been mentioned in the Australian association of Health care standards, such that each surveyor has a specific profile in the database. The surveyor’s profile is reviewed prior to their placement in the assessment team to ensure that they match the details of their file. The details include the role of the surveyor in assessment and the changes that have occurred over the past 2 years. In addition, the database profile contains details of the surveyor’s competence, experience, and expertise (15).

One of the limitations of the present study was access to key informants and experts and gaining their cooperation to participate, due to their liability, occupation, and value of their work time.

In general, effective surveyor management of the hospital accreditation program will be capable of increasing the quality and safety of healthcare by increasing the reliability and validity of accreditation results. For the first time in Iran, the present study identified the dimensions and variables affecting surveyor management of hospital accreditation, classified them on the basis of existing scientific methods, and provided a conceptual framework. We can definitely exploit these findings to effectively manage surveyors of accreditation programs and take a big step toward increasing survey reliability.