Quality improvement competency gaps in primary care in Albanian, polish and slovenian contexts: a study protocol.

BACKGROUND: Nowadays, general practitioners (GPs) and family doctors (FDs) face increasing demands, as a consequence of complex patients' expectations, developments in science and technology, and limitations within healthcare systems which can result in competency gaps. Therefore, there is a need to identify which competencies in quality improvement (QI) are most important for GPs and FDs to possess in order to meet the demands of contemporary health care practice. To date, however, little information is available on the self-assessment of competencies related to QI among GPs and FDs. To deal with these issues, a project on QI in continuous medical education was launched in 2011. The project aims to broaden the GPs'/ FDs' continuous education offer, its quality and attractiveness, as well as provide them with opportunities for vocational advancement and enable the development of common, European frame of reference for GPs'/FDs' occupational competencies. The third work package of the project consists of the validation research of the questionnaire developed on the basis of the competency framework in QI for GPs/FDs in Europe.
METHODS: A cross-sectional study will be carried out using the self-assessment QI questionnaire which was originally developed in English and subsequently it was cross-culturally adapted in Slovenian, Albanian and Polish settings by use of a pilot study on a conveniently selected group of FDs/GPs (N=10) in each participating country. The final version of the questionnaire will be administered to large samples in each country involved in the survey. Two weeks after the first administration of the questionnaire, a second round, with the same procedure and including the same group of respondents, will follow. Psychometric tests will be conducted including internal consistency (after the initial and subsequent application of the instrument) and stability over time (two-week test-retest reliability). DISCUSSION: This self-assessment study will demonstrate the complex environment in which general practice/family medicine operates and, eventually, this gap analysis will set out strategically important areas for collaborative efforts related to QI in primary care. The authors consider that the study should be extended to other European countries to help identify most required competencies that GPs/FDs should possess in Europe and thus stir system and educational debate around QI curricula and training for primary care in Europe.

BACKGROUND

Modern medicine recognises that the outcomes of clinical care not only depend on how doctors put their clinical knowledge and training into practice, but also on other skills such as being able to deal with the continuous flow of new information, emerging medical evidence and effective management of available resources (1). Today, family doctors (FDs) face increasing demands, as a consequence of complex patients’ expectations (2), developments in science and technology, and limitations within healthcare systems (3) which can result in competency gaps. This means that FDs of today need to be able to reflect on the organizational systems in which they deliver care and, as needed, effectively participate in changing those systems to improve the quality of care (4). Quality improvement has been recognized as an important part of continuous medical education (CME) by several important parties in education and quality since at least 2001. Therefore, there is a need to identify which competencies in quality improvement (QI) are the most important for general practitioners (GPs) and FDs to possess in order to meet the demands of contemporary health care practice. Various assessment tools of good quality (5) have been developed to assess doctors learning processes and their competences at the authentic workplace (6). Little information is available on the self-assessment of competencies related to QI among primary care physicians, family medicine (FM) teachers and primary care patients.

To deal with the above issues, a project on QI in CME was launched in 2011.

Introduction to the project

GPs’ and FDs’ involvement in the cardiovascular diseases (CVD) “first-line” treatments, prevention and health promotion is a problem of the highest importance in European public health (PH) (7, 8). First stages of the treatment of patients with chronic diseases (especially, CVD) might be improved by new approaches to lifelong learning (LLL) of GPs and implementing modern information technology (IT) in existing educational systems in Europe (9). This study is a part of a large European project entitled “Innovative lifelong learning of European General Physicians in Quality Improvement supported by information technology” (InGPinQI) No. 2010-1-PL1-LEO05-11473 financed by European Commission Lifelong Learning Program which consists of ten work packages (WPs) (Figure 1). The InGPinQI project is focused on the improvement of the QI education in hypertension and diabetes mellitus, which are important problems of public health in Central and Eastern Europe (10, 11, 12, 13). It aims to improve the existing continuous professional development (CPD) programs for GPs/FDs and teachers in family medicine and their competencies in the field of QI. This might be achieved by implementing innovative tools (e.g. a web-based tool measuring the required competencies and educational needs, a distance-learning course for GPs/FDs) in existing educational systems in Europe supported by IT (Figure 2).

Figure 1

Overview the European project inGPinQI No. 2010-1-PL-1-LE005-11473. The training materials (WP4, WP5, WP7) and the courses (WP5, WP6) developed in the project are based on the data of the identified gaps in the educational systems (WP1, WP2) and required GPs competencies (WP3) relating to the vital problems in the european communities, namely: (a) inequalities in health and social determinants of health, (b) growing patient expectations towards quality of services, (c) fragmentary GP’s education on QI, (d) unused expert knowledge of physicians, (e) growing demand on cross-border services.

Figure 2

Tangible products developed in the inGPinQI project No. 2010-1-PL1- LE005-11473.

AIM AND OBJECTIVES

The project aims to broaden the GPs’/FDs’ educational offer and its quality and attractiveness, as well as provide GPs/FDs with opportunities for vocational advancement and enables developing of common, European frame of reference for GPs’/FDs’ occupational competencies.

The third work package (WP 3) of the project consists of the validation research of the developed questionnaire on the basis of the competency framework in QI for GPs/FDs in Europe. The development of the framework has been extensively described elsewhere (14). The WP 3 has two main objectives:

To validate the QI self-assessment questionnaire and;

To identify the competency gaps between currently possessed competencies by GPs/FDs and those which are most required to be possessed by them to provide high level patient-centred care in Slovenian, Polish and Albanian health care context.

Emphasis is placed on:

Current self-perceived level of competencies related to QI possessed by GPs/FDs in Poland, Slovenia and Albania and, additionally, perceived level of GPs/FDs’ competencies from the perspective of patients (in Albania only).

The desirable/expected level of QI competencies that GPs/FDs should possess measured from the perspective of FM teachers in Slovenia, Poland, and Albania and, additionally, from the perspective of the decision-makers (in Albania only).

In addition, the gap or mismatch between the currently possessed competencies and the required ones is being identified and recommendations are being made on the development and introduction of the educational interventions in QI for GPs/FDs.

The aim of this article is to present the study protocol on the validation of the QI self-assessment questionnaire. Additional aim is to get first results on the self-perceived level of QI competencies from the perspective of the GPs/FPs, FM teachers, patients and policy makers which will to identify priority areas within the general/family practice environment in Albanian, Slovenian and Polish context.

Procedures of the validation of the QI self-assessment questionnaire

QI self-assessment questionnaire

The questionnaire is based on the QI Competency framework for GPs/FDs in Europe (14). It consists of two parts: 1) assessment of competencies, and; 2) demographic information which varies depending on the target group of respondents who are GPs/FDs, FM teachers and trainers, patients and policy makers.

Part 1 of the questionnaire includes 6 comprehensive and interdisciplinary competency domains: Patient Care & Safety, Effectiveness & Efficiency, Equity & Ethical Practice, Methods & Tools, Leadership & Management, and Continuing Professional Education. Thirtyfive specific competencies are organized into the 6 domains. We used a five-stage Dreyfus scale (15) which proved useful when thinking about medical education (15). It consists of five categories: novice (1 point), advanced beginner (2 points), competent (3 points), proficient (4 points), and expert (5 points). The summary score of the whole questionnaire ranges from a minimum 35 points to a maximum 180 points. The minimum and maximum summary scores of the six domains range as follows: Patient Care & Safety (8-40 points), Effectiveness & Efficiency (7-35 points), Equity & Ethical Practice (8-40 points), Methods & Tools (5-25 points), Leadership & Management (4-20 points), and Continuing Professional Education (5-25 points).

The second part of the questionnaire consists of:

For GPs/FDs: gender, age, type of practice, whether it is public or private entity, the catchment area of the practice (number of inhabitants and rural/urban), number of years served in practice, remuneration system and whether they are involved in teaching activities;

For FM teachers/trainers: age, the catchment area of the practice (number of inhabitants and rural/urban), years of experience as family medicine teachers, specialization, type/level of training provided, position in education;

For patients: gender, age, current life situation, education status, number of times of seeing their family doctor in the period of past 12 months;

For decision-makers: age, place of residence (number of inhabitants and rural/urban), sector they work for, job title.

Study design

Three centres will be involved in the validation study: Institute for Development of Family Medicine with cooperation of Departments of Family Medicine Maribor and Ljubljana Medical School, Slovenia (SL), College of Family Physicians in Poland with cooperation with the Department of Family Medicine, Jagiellonian University Medical College (PL) and Department of Public Health, University of Tirana, Albania (AL).

The study aims to validate the questionnaire and to identify the existing gaps between the current/self-perceived level of competencies related to QI possessed by GPs/FDs and the desirable/expected level of competencies by FD teachers FDs should possess to provide high level patient-centred care. In order to do so, several stages of the study are foreseen.

Stages A & B of the study will constitute of a validation exercise as described below:

The current/self-perceived level of competencies will be measured from the perspective of GPs/FDs in Poland and Slovenia and from the perspective of GPs/FDs and patients in Albania by using a self assessment QI questionnaire developed in the framework of InGPinQI Project (Figure 1).

The desirable/expected level of competencies will be measured from the perspective of FM teachers/trainers in Slovenia and Poland and from the perspective of FM teachers/trainers and decision-makers in Albania.

Validation Methods

This cross-sectional study will be carried out as postal survey using the self-assessment QI questionnaire which has been translated back and forth from English to Slovenian, Albanian and Polish and piloted on a conveniently selected group of FDs/GPs (N=10) in each participating country (and patients in Albania). Furthermore, in three countries, the respective versions of the questionnaires have been language and culturally adapted (16, 17, 18). We will adapt the questionnaire based on the suggestions from the pilot study.

The final version of the questionnaire will be then sent to the study group, described below. To assure the adequate response rate, the first reminder will be sent one week after the first round. Two weeks after the first questionnaire will be sent, the second round with the same procedure will follow. In case of no response, the researchers will try to contact the respondents by phone if the conditions allow. In case of too low response rate, focus groups will be arranged with a group of respondents which does not reach a required quota.

The study will be carried out in stages in Slovenia, Poland and Albania. For stage A & B, 2 psychometric tests will be used: internal consistency (after the initial application of the instrument) and stability over time (after reapplication of the test after 2 weeks to the same group of respondents) [see “Statistical analysis” section for further details].

Ethical considerations

The study will be performed in line with the Helsinki Declaration using survey. The participants will be informed about the goal and objectives of the study and will be given a possibility to opt out without giving explanation if by any chance they feel uncomfortable with the content of the questions. The information about the research will be posted together with the questionnaire. In case of noresponse, the researchers will seek to get a proper consent from the respondents to contact them by the phone. If the focus group has to be arranged with a group of respondents which does not reach a required quota, an informed consent form will be developed to be signed by the respondents. The ethical approval has been applied for and granted by: the National Ethics Committee in Slovenia (No. 96/05/12), the National Committee of Biomedical Ethics in Albania. In Poland the study will be conducted under the terms of all relevant local and national legislation. Approval by Ethical Committee is not necessary because the study has not experimental design and will not involve patients. Moreover, protocol conforms to the ethical guidelines of the 1975 Helsinki Declaration.

Stage A: Current/self-perceived level of QI competencies self-assessment (GPs’/FDs and patients’ perspective

Sampling

The respondents for Stage A: GPs/FDs and patients (only in Albania) will be randomly selected based on the general country register for GPs and GP practice records (for patients). In Poland, the database of the College of Family Physicians will be used. In Slovenia, the database of the Slovenian Family Medicine Society will be used. We aim to achieve N=100 responses from GPs/FDs (and N=100 patients in Albania). In the pre-test phase, a convenient sample of 50 FDs and 100 patients will be recruited in Tirana only.

After the validation exercise, the actual survey will include 2000 patients in 4 districts of Albania (Tirana, Shkoder, Elbasan and Vlore). The sample will be randomly selected based on the GPs’ lists in each of the districts included in the survey. As for the GPs, the questionnaire will be administered to a random sample of GPs operating in primary health care centres in the 4 districts included in the survey.

The details of the study in stage A in each centre are presented below:

Pre-test: Application of the questionnaire on self-perceived level of QI competencies from GPs/FDs perspective (N=100):

Calculation of internal consistency;

Reapplication of the questionnaire to the same groups of respondents after 2 weeks (calculation of test-retest reliability).

Stage B: Expected/required level of QI competencies possessed by GPs from FM/GP teachers and decision makers perspective

The respondents for Stage B:FM/GP teachers and decision makers will be randomly or conveniently selected based on the general country register for GP/FD teachers and professional contacts for decision makers. In Slovenia, the database of the Slovenian Family Medicine Society will be used. We aim to achieve N=50 responses from GP/FD teachers and N=50 from decision makers in respective centres.

In Poland in order to select the randomized sample of FM teachers, approx. 10 centres providing educational programmes for GPs/FDs (from the total number of 50 institutions) will be randomly selected and invited to the study (a phone call with directors). If the response with a positive attitude will be insufficient, the sample will be complemented by a further number of randomly chosen institutions. 3-4 questionnaires will be sent by a regular mail to those centres which will be interested in participating in the study. Directors of those centres will indicate the respective teachers who should be involved in the study. If the response rate is insufficient they will be contacted by phone.

The details of the study in stage B in each centre are presented below:

Pre-test: Application of the questionnaire on expected level of QI competencies in GPs from FM/GP teaches perspective (N=50):

Calculation of internal consistency;

Reapplication of the questionnaire on the same groups of respondents after 2 weeks (calculation of test-retest reliability)

Statistical analysis

Cronbach’s alpha will be used to calculate the internal consistency of the questionnaire on self-perceived level of QI competencies from GPs/FDs perspective (stage A of the validation study) and of the questionnaire on expected level of QI competencies in GPs/FDs from FM/GP teaches perspective (stage B of the study). Cronbach’s alpha will be calculated for the whole questionnaire and for each of the six domains.

Spearman’s who will be used to measure test-retest reliability (stability over time) of the questionnaire on self-perceived level of QI competencies from GPs/FDs perspective (stage A of the validation study, during which the instrument will be reapplied after 2 weeks to the same group of respondents) and of the questionnaire on expected level of QI competencies in GPs/FDs from FM/GP teaches perspective (stage B of the study, during which the instrument will be reapplied after 2 weeks to the same group of respondents). The test-retest reliability of the summary score will be calculated for the whole questionnaire and for each of the six domains.

EXPECTED RESULTS AND DISCUSSION

With the planned study, we will validate the newly developed competency framework questionnaire (19) in Polish, Albanian and Slovenian language. We expect that the questionnaire will have good reliability and time stability. Based on this questionnaire, we will also get first data on the self-perceived level of QI competencies from the perspective of the GPs/FPs, FM teachers, patients and policy makers specifically focusing on the validity and reliability of the used instrument. This will enable us to identify priority areas within the general practice/family medicine environment in Albanian, Slovenian and Polish context. We expect the study will point to differences between the self-perceived competencies by GPs/FDs and between the desired competencies by FD teachers and trainers. Based on this gap, we will be able to identify the weakness which could affect aspects of the quality of care being provided. Therefore, this study can contribute to the development of recommendations for priority actions related to these gaps. Identification of the gaps between possessed and expected/required level of competencies based on the results per centre will enable us to get a first idea on the recommendations for educational interventions needed in QI education/training for GPs/FDs. The results of the study will allow comparisons among participating countries with acknowledging specificities of different health care systems (20, 21).

This self-assessment study will demonstrate the complex environment in which general practice/family medicine operates and this gap analysis will set out strategically important areas for collaborative efforts related to QI in primary care. The authors believe that the study should be extended to other European countries to help identify most required competencies that GPs/FDs should possess in Europe and thus stir system and educational debate around QI curricula and training for primary care in Europe.