Cross-cultural Adaptation of a Questionnaire on Self-perceived Level of Skills, Abilities and Competencies of Family Physicians in Albania.

OBJECTIVE: Our aim was to validate an international instrument measuring self-perceived competency level of family physicians in Albania.
METHODS: A representative sample of 57 family physicians operating in primary health care services was interviewed twice in March-April 2012 in Tirana (26 men and 31 women; median age: 46 years, inter-quartile range: 38-56 years). A structured questionnaire was administered [and subsequently re-administered after two weeks (test-retest)] to all family physicians aiming to self-assess physicians' level of abilities, skills and competencies regarding different domains of quality of health care. The questionnaire included 37 items organized into 6 subscales/domains. Answers for each item of the tool ranged from 1 ("novice" physicians) to 5 ("expert" physicians). An overall summary score (range: 37-185) and a subscale summary score for each domain were calculated for the test and retest procedures. Cronbach's alpha was used to assess the internal consistency for both the test and the retest procedures, whereas Spearman's rho was employed to assess the stability over time (test-retest reliability) of the instrument.
RESULTS: Cronbach's alpha was 0.87 for the test and 0.86 for the retest procedure. Overall, Spearman's rho was 0.84 (P<0.001). The overall summary score for the 37 items of the instrument was 96.3±10.0 for the test and 97.3±10.1 for the retest. All the subscale summary scores were very similar for the test and the retest procedure.
CONCLUSION: This study provides evidence on cross-cultural adaptation of an international instrument taping self-perceived level of competencies of family physicians in Albania. The questionnaire displayed a satisfactory internal consistency for both test and retest procedures in this sample of family physicians in Albania. Furthermore, the high test-retest reliability (stability over time) of the instrument suggests a good potential for wide scale application to nationally representative samples of family physicians in Albanian populations.

INTRODUCTION

Public expectations relate to a good quality of health care services and, therefore, the general public and especially patients require health care professionals to provide high quality health care services. Quality of health care is defined as the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge (1). In recent years, quality improvement and performance evaluation have become core issues in health care practice. Therefore, quality improvement needs to be included at all levels of medical education and in all aspects of health care services. This approach implies that medical students at all levels should not only receive medical knowledge during their training, but should also acquire abilities, skills and competencies in the areas of quality improvement and quality assurance (2).

In family medicine or general practice, quality of health care includes different aspects and covers a wide array of physicians’ scope of work such as a comprehensive/holistic approach to health care provision, management skills, community orientation, problem-solving skills and abilities, as well as an individual-centered approach to health care services (3).

Thus, it is widely recognized that competencies in quality improvement are crucial for family physicians and general practitioners in order to foster patient care. From this perspective, specific roles, abilities, skills and competences have been defined at all training levels for medical doctors including also continuing medical education (4). These roles and competences are classified in frameworks such as the CanMEDS (Canadian Medical Education Directives for Specialists) Roles Framework (5,6), the six core competences identified and described by the Accreditation Council for Graduate Medical Education (ACGME competencies) (7) and Tomorrow Doctor’s at the UK (8). Models of abilities, skills and competencies are also deemed as a useful tool for self-assessment of primary health care physicians committed to practice-based learning (9,10) who want to improve their health care practices, analyze their clinical experience, plan improvement strategies, and determine a putative improvement incorporating knowledge into the habitual daily practice (10).

However, little is known about the inclusion, content and outcomes of teaching quality improvement topics within the medical curricula in European countries, especially in the post-communist countries of Central and Eastern Europe including Albania.

In this framework, the aim of this study was to validate an international instrument (developed with the support of the European Community Lifelong Learning Program) regarding self-perceived level of skills, abilities and competencies of family physicians operating in primary health care services in Albania, a post-communist country in Southeast Europe.

METHODS

A representative sample of 57 family physicians operating in primary health care system in Tirana city was interviewed in March-April 2012 (49 males and 65 females; mean age: 45.8±10.1 years). All family physicians were asked to self-assess their level of skills, abilities and competencies regarding the following domains of quality of care:

Patient care and safety (8 items);

Effectiveness and efficiency (7 items);

Equity and ethical practice (8 items);

Methods and tools (5 items);

Leadership and management (4 items), and;

Continuing professional development (5 items).

Answers for each item of each subscale ranged from 1 (“novice”= physicians have little or no knowledge/ability, or no previous experience of the competency described and need close supervision or instruction) to 5 (“expert”=physicians are the primary sources of knowledge and information in the medical field).

An overall summary score (including 37 items; range: 37-185) and a subscale summary score for each domain were calculated for all family physicians for both the test and the retest procedures. Data on work experience, specialization of family physicians and their involvement in teaching and training activities were also collected.

Cronbach’s alpha was used to assess the internal consistency of the full scale (37 items) and each of the 6 subscales, separately for the test and the retest procedures. Spearman’s rho was used to assess the test-retest reliability (stability over time) for the overall scale and each subscale of the instrument.

RESULTS

In this representative sample of 57 family physicians in Tirana (26 men and 31 women), median age was 46.0 years (interquartile range: 37.5-56.0 years) [Table 1]. Two thirds were general practitioners and the rest included specialized physicians (in Cardiology, Gynecology, Rheumatology, etc.). Median time-span of work experience (years of practice) was 23 years (interquartile range: 14-31 years). Only 28% of family physicians included in this validation study were involved in teaching activities with medical students, or other types of training activities.

Table 1

Distribution of socioeconomic characteristics, specialization status and work experience in a sample of family physicians (N=57) in Tirana, Albania, 2012 * Median values and interquartile ranges (in parentheses). † Numbers and column percentages (in parentheses).

The internal consistency of the whole scale (37 items) was Cronbach’s alpha=0.87 for the test and Cronbach’s alpha=0.86 for the retest (Table 2). For both the test and the retest procedures, Cronbach’s alpha was the lowest for “leadership and management” subscale (0.79 for the test and 0.77 for the retest) and the highest for the “effectiveness and efficiency” domain (0.95 and 0.93, respectively).

Table 2

Internal consistency of each domain (subscale) of the instrument for the test and retest application in a representative sample of family physicians in Albania (N=57) * Cronbach’s alpha.

Overall, the instrument exhibited a good stability over time: Spearman’s rho=0.84, P<0.001 (Table 3). The testretest reliability was the highest for the “patient care and safety” subscale (rho=0.96, P<0.001) and the lowest for the “leadership and management” domain (rho=0.67, P<0.001).

Table 3

Stability over time (test-retest reliability) for each domain (subscale) of the instrument

The overall summary score for the 37 items of the instrument was 96.3±10.0 for the test and 97.3±10.1 for the retest (Table 4). All the subscale summary scores were very similar for the test and the retest procedure.

Table 4

Summary score of each domain (subscale) of the instrument for the test and retest application * Mean values ± standard deviations.

DISCUSSION

On the whole, this international tool showed a reasonable internal consistency for both test and retest procedures in this representative sample of family physicians operating in primary health care services in Tirana, the Albanian capital. However, Cronbach’s alpha was not very high for the “leadership and management“ subscale. Conversely, the internal consistency was very high (Cronbach’s alpha≥0.90) for most of the domains.

Overall, the questionnaire showed a satisfactory stability over time. But again, the test-retest reliability was not high for the “leadership and management” domain, which may point to the need for further tuning of this subscale of the instrument.

The questionnaire self-assessing the level of skills, abilities and competencies of family physicians was designed in line with the Quality Improvement Competency Framework (QICF) which, in turn, was developed in the course of a systematic consensus study carried out among European primary care experts interested or specializing in quality improvement area (10). The QICF is structured into six domains aligned with the questionnaire which we report in this article for self-assessment of skills and competencies of family physicians and general practitioners: Patient Care and Safety, Effectiveness and Efficiency, Equity and Ethical Practice, Methods and Tools, Leadership and Management, and Continuing Professional Education (10). Each of the domains reflects an important dimension of the quality of health care. The domains of the instrument require implicitly reflection and self-assessment in order to improve the quality of health care provision (9). Each domain of the instrument measures a number of specific competencies which constitute individual standards (10). This study provides evidence of a cross-cultural adaptation of an international questionnaire taping self-perceived level of skills, abilities and competencies of family physicians operating in the primary health care services in a transitional Southeast European population. In conclusion, in the Albanian context we validated a useful tool measuring family physicians’ self-perceived level of competencies regarding different domains of health care services. After the validation procedures reported in this article, this instrument is currently being administered to a large representative sample of family physicians working in primary health care services in different districts of Albania.