The Validity and Reliability of the Turkish Version of Jefferson Scale of Physician Lifelong Learning for Medical Students.

OBJECTIVE: Lifelong Learning is a voluntary and self-motivated form of learning that lasts from birth to death for personal and professional reasons. As medical science has a dynamic nature, the knowledge gained in the faculty of medicine mostly will not be enough in the later professional life. The aim of this study is to determine the validity and reliability of the JeffSPLL-MS© in Turkish.
METHOD: Linguistic equivalence analysis was applied first, and confirmatory factor analysis (CFA) was utilized in order to test the construct validity. Concurrent validity was tested by simultaneous administration of Lifelong Learning Trends Scale (LLTS) and JeffSPLL-MS©. Finally, internal consistency and test-retest reliability of the scores gathered from the scales in the study were calculated.
RESULTS: Linguistic equivalence study supplied that there is a strong relationship (r=.873, p=.001) between Turkish and English forms of the scale. For CFA, Kaiser Meyer Olkin (KMO) value was .863 and the results of Bartlett Sphericity Test were appropriate (X2 = 1173,6; p=.001, df=91); model-data fit indices for JeffSPLL-MS© with 14 items in three factors were also satisfactory ((X2/df =1,51); RMSEA=0,046; NFI=0,918; RFI=0,902; CFI=0,971) and factor loadings of items were ranging between.522 and.764. The Pearson correlation coefficient as the indicator of concurrent validity of the scales LLTS and JeffSPLL-MS© was calculated as .624 (p=.001). The internal consistency (Cronbach alpha) of the total scores gathered from JeffSPLL-MS© is .843 and stratified alpha coefficient is .892. Cronbach alpha values for the subscales are as follows: .830 (F1=Learning beliefs and motivation), .719 (F2=Skills in seeking information), .721 (F3=Attention to learning opportunities). The test-retest reliability values for the scale, and its subscales were ranged between.709 and.812 within a four-week period.
CONCLUSION: It is concluded that JeffSPLL-MS© can be used as a valid and reliable measurement instrument for medical education studies in Turkey. © Copyright Istanbul Medeniyet University Faculty of Medicine.

Introduction

Lifelong Learning (LL) is a voluntary and self-motivated form of learning that lasts from birth to death for personal and professional reasons. This type of learning includes all formal or non-formal learning activities that are not specific for childhood or not limited to regular classrooms[1]. The concept of LL first began to develop in the 1980s. LL activities related to learning in Turkish universities began in the same year. Currently, LL is the focus of almost all education systems globally[2-4].

Medical science has a dynamic nature. Diagnostic criteria and treatment protocols are changing. Since new medical information is obtained as a result of medical researches, these changes are evolving very rapidly along with technological developments in the field of medicine[5]. For this reason, the knowledge gained in the faculty of medicine mostly will not be enough in the later professional life. LL is very important for medical students to follow new medical researches and obtain new information and upgrade themselves, and as a result, to provide a much better patient care[6,7]. Therefore, teaching and motivating the medical student’s dedication to learning through their entire life is an important part of medical education and an important responsibility of educators[8,9]. Throughout medical education, students should be provided with LL skills, information literacy, learning how to learn and how to search through information resources, and become lifelong learners[8].

A number of scales have been developed to identify physicians’ lifelong learning tendencies. The Jefferson Scale for Physician Lifelong Learning (JeffSPLL), created by Hojat et al., has been shown to be one of the most successful among them[10,11]. JeffSPLL-MS, the adapted version of this scale for medical students (MS), measures the orientation of medical students regarding LL[8,11].

To measure the LL tendencies of physicians in Turkey, there are unfortunately very few scales adapted into Turkish by validity and reliability studies. Two of these are ‘the Lifelong Learning Tendency Scale (LLTS)1 (a valid and reliable scale for LL), which was adapted for the field of medical education by collecting data from physicians by Arslan et al[12] and ‘the Jefferson Scale of Physician Lifelong Learning’[13]. However, there is no scale developed for the LL trends of medical students in Turkey or translated into Turkish by conducting a validity and reliability study, except this current one.

In this study, data were collected using two scales simultaneously. One is JeffSPLL-MS, and the other is the adapted version of LLTS for the medical students, which is also used to check the concurrent validity of JeffSPLL-MS. The aim of this study was to determine the validity and reliability of the JeffSPLL-MS in Turkish, and the LL trends of the medical students and whether they have the motivation and competence related to this. In this way, it will be possible to know more about LL in medical education in Turkey. It may also be useful to compare the results with those of the medical students in other countries.

Method

Participants and Procedure

Among a total of 802 students, 249 (31%) attending to their 1st through 3rd years of medical education at Istanbul Medeniyet University School of Medicine who volunteered to participate were enrolled to the present single-centered study. For data collection, the questionnaires were sent to all of the 802 students online, and only 249 students answered the questionnaires. Most of the responders (97%) invited for enrollment on-line were in their 2nd and 3rd year. Data collection phase was performed between February and May 2019.

First, the original English version of the scale was translated into Turkish by two English language experts. Later, two different English-Turkish language experts translated the scale back to English. After the reverse translation procedures, in order to determine the linguistic suitability of the scale, English and Turkish versions of the scale were administered within two weeks period in May 2019 to 31 (18 female and 13 male, mean age 20.7) students whose native language is Turkish and who have also sufficient English language proficiency. All of them were registered to the School of Medicine as medical students and volunteered to participate in the study.

Instruments

Demographic Information Form: It was developed by the researchers within the scope of the research in order to collect some demographic information of the participants.

Lifelong Learning Tendency Scale (LLTS): The scale was developed in 2009 by Yelkin Diker Coşkun as part of her doctoral thesis[1]. It consists a total of 27 items that are clustered in four sub-scales, namely: 1-Motivation (6 items), 2-Persistence (6 items), 3-Lack of learning regulation (6 items), and 4-Lack of Curiosity (9 items). Scale items are evaluated with a 6-point Likert scale from “It fits very well” to “It never fits”. The scales of “Lack of learning regulation” and “Lack of Curiosity” are scored in reverse. In addition to sub-scale scores, LLTS also provides a total test score. This scale was adapted for medical education by collecting data from Turkish physicians by Arslan et al.[12]. Cronbach alpha reliability coefficient was reported as .89 for the total scale score, and values for the subscales were not reported in the original study. The factor loads of the items ranged from 0.36 to 0.74. Since no other scale available in Turkish to measure the LL tendencies in medical education, this scale was used in this study to determine the concurrent validity of the Jefferson Scale of Physician Lifelong Learning Scale - Medical Students Version (JeffSPLL-MS). Reliability coefficients in this study are presented in Table 2.

Table 2

Findings regarding the reliability of scores obtained from JeffSPLL-MS and LLTS and their subscales

Scale-Subscale	Item Numbers	Cronbach α (n=249)	Stratified α (n=249)	Test-retest (n=71)
JeffSPLL-MS	14	0.843	0.892	0.674
Learning Beliefs and Motivation	8	0.830		0.645
Skills in Seeking Information	3	0.719		0.609
Attention to Learning Opportunities	3	0.721		0.611
LLTS	27	0.905	0.923	0.712
Motivation	6	0.864		0.675
Persistence	6	0.861		0.649
Lack of Learning Regulation	6	0.823		0.625
Lack of Curiosity	9	0.885		0.701

All correlations are significant at 0.001 level.

The Jefferson Scale of Physician Lifelong Learning Scale - Medical Students Version (JeffSPLL-MS): Originally named as “Jefferson Scale of Physician Lifelong Learning (JeffSPLL)”, this measure was designed to determine the lifelong learning tendencies of physicians by Mohammadreza Hojat et al.[13]. In 2010, it was adapted for medical students by Wetzel et al.[11], and named as Jefferson Scale of Physician Lifelong Learning for Medical Students (JeffSPLL-MS). It consists of 14 items, clustered in three factors. The headings of the subscales are: Factor 1- Learning beliefs and motivation (8 items), Factor 2- Skills in seeking information (3 items) and Factor 3- Attention to learning opportunities (3 items). Items of the scale are scored with a 4-point Likert rating such as “Strongly Disagree”, “Disagree”, “Agree”, “Strongly Agree”, and there are no reverse scored items in the scale. In addition to subscale scores, it also provides total test scores. Cronbach alpha internal consistency reliability coefficient based on data collected from 652 medical students was 0.77 for the total scale score, values for subscales were 0.70 (factor 1), 0.61 (factor 2) and 0.59 (factor 3), respectively, and item factor loads were reported at values ranging from 0.73 to 0.32. It is concluded that the scale was a valid measurement instrument at the acceptable level that able to yield reliable results. Reliability coefficients in this study are presented in Table 2.

Prior to the study, necessary permissions were obtained from the Ethics Committee, Hojat Mo-hammadreza, and Jefferson University to adapt the scale into Turkish.

Analysis

IBM SPSS 26 and AMOS 26 programs were used for the statistical analysis of the data in this study. First, demographic information and distribution of the data obtained with JeffSPLL-MS and LLTS and descriptive statistics were examined by using SPSS. For construct validity, confirmatory factor analysis (CFA) in AMOS was used and SPSS was used again to calculate the reliability of the data obtained from the measures. After collecting the data, Kaiser Meyer Olkin (KMO) sampling adequacy coefficient was calculated as .863 and Bartlett Sphericity Test was applied (χ2 = 1173,6; p=0.001, df=91) to determine the suitability of the data for CFA and their results were found appropriate for the analysis. Three factors in the Jef-fSPLL-MS are as follows:

Learning beliefs and motivation (F1)

Skills in seeking information (F2)

Attention to learning opportunities (F3)

CFA was applied to determine the construct validity of the scale. AMOS version 26 was utilized for CFA, and the main model-data fit indices (goodness-of-fit measures) in the AMOS program for testing model-data compatibility in CFA are: chi-square (χ2) / degree of freedom (df), RMSEA, NFI, RFI, CFI, and TLI. In order to examine the validity of the scale, adapted version for medical education of a similar scale named as Lifelong Learning Tendency Scale (LLTS) was applied to the participants simultaneously. The suitability of the data obtained from the participants to the measurement models was thoroughly examined. The data fit indices of the JeffSPLL-MS measurement model were tested with CFA and were found as follows: X2/df=1.51 (<2, Good Fit), RMSEA=0.046 (<0.05, Good Fit), NFI=0.918 (>0.9 & <0.95, Acceptable Fit), RFI=0.902 (>0.9 & <0.95, Acceptable Fit), CFI=0.971 (>0.95, Good Fit), TLI=0.970 (>0.95, Good Fit). Considering the “Good Fit” and “Acceptable Fit” ranges[14] as stated, it was concluded that the data sufficiently fitted the model.

Reliability means that the results obtained from a measurement model are consistent and free from measurement errors to a certain extent. While there are a variety of methods for determining reliability values, the most commonly used methods are internal consistency calculations based on inter-item covariance matrices. In order to determine the internal consistency reliability values of JeffSPLL-MS, LLTS and their subscales, internal consistency Cronbach α coefficients are calculated. It is recommended to use the stratified α coefficient and compare the values with Cronbach α values to make conclusions about the reliability of the total test scores obtained from multidimensional tests such as JeffSPLL-MS and LLTS[15]. It has been stated that internal consistency reliability coefficient can be better determined by the stratified α coefficient of the scores obtained from multidimensional tests since Cronbach α values are accepted as the lower limit. Therefore, it is recommended to calculate the stratified alpha coefficient to determine reliability in multidimensional measurement tools such as JeffSPLL-MS[15]. In addition to the internal consistency reliability, approximately one month (four weeks) after the first application date of the scale, measures were re-applied to 71 (40% of the first application group) and 51 29% of the first application group) participants. Appropriate levels of test-retest reliability coefficients of the subscales were also determined.

Results

Hundred and ten (41%) participants were male, and 139 (59%) were female students; their ages varied between 18 and 25, and the average age was 21.2. The ages of the participants ranged between 18 and 25 years, with the average age as 21.18 years.

Linguistic Equivalence

Parallel forms of the JeffSPLL-MS in Turkish and English were applied to 31 participants who spoke both English and Turkish fluently (participants whose native language is Turkish and also have sufficient English language proficiency) at two-week intervals and the results were compared. A strong and significant relationship was found between them r=0.873 at the statistical significance level of p <0.001.

Construct Validity

For construct validity of the scale, CFA was used as stated in the analysis section, and standardized coefficients related to JeffSPLL-MS measurement model are given in Figure 1. When the factor loadings of the items in the JeffSPLL-MS measurement model were examined, it was understood that the highest item loads were found in the “Skills in seeking information” (F2) factor with the factor load as high as 0.76. It is the 4th and the 14th items that are with the lowest factor loads on the scale (0.52). When the covariances among the factors were analyzed, it was understood that the highest covariance was between the factor “belief motivation learning” (F1) and the factor “evaluation of learning” (F3). The correlation value of r=0.425 between these two factors was significant at p<0.001. Some covariance modifications were also required in CFA according to high model-data compatibility. The most frequent covariance modifications were among the error terms of the items within the same factor. For correlations among all items, they were all significant at the level of p=0.001 and higher than r=0.346.

Figure 1

JeffSPLL-MS construct validity tested by CFA (standardized coefficients).

Concurrent Validity

Concurrent validity can be used to support the construct validity. For this purpose, LLTS was administered as an only available similar scale in Turkish for JeffSPLL-MS. In Figure 2, the standardized coefficients for this measurement model are provided. Considering the model fit indices (goodness-of-fit measures), χ2/df=1.76 (<2, Good Fit), RMSEA=0.056 (>0.05 & <0.1, Acceptable Fit), NFI=0.906 (>0.9 & <0.95, Acceptable Fit), RFI=0.901 (>0.9 & <0.95, Acceptable Fit), CFI=0.952 (>0.95, Good Fit), TLI=0.930 (>0.9 & <0.95, Acceptable Fit), it can be concluded that the research data are well fitted with the LLTS measurement model. Most of the values were found at an “acceptable” level, and some of them were fitted fairly good. Therefore, LLTS was used to test the validity of similar scales.

Figure 2

Factor structure of LLTS tested by CFA (standardized coefficients).

In this study, data were collected using two scales simultaneously. One is JeffSPLL-MS, and the other is LLTS, which is used to check the concurrent validity. High level relationships are expected between the subscales of these two scales. The data were distributed normally, and the Pearson correlation coefficients among the subscales are listed in Table 1. Significant relationships between both scale substructures were observed (p<0.001). It was observed that there were negative and strong relationships between LLTS subscales named as “lack of learning regulation” and “lack of curiosity” and other subscale scores.

Table 1

Pearson correlation coefficients between JeffSPLL-MS and LLTS subscales.

	JeffSPLL-MS			LLTS
	1-Learning Beliefs and Motivation	2-Skills in Seeking Info.	3-Attention to Learning Opportunities	4-Motivation	5-Persistence	6-Lack of Learning Regulation	7-Lack of Curiosity
1	1
2	0.196	1
3	0.425	0.345	1
4	0.587	0.358	0.212	1
5	0.294	0.430	0.302	0.742	1
6	-0.386	-0.354	-0.351	-0.375	-0.305	1
7	-0.302	-0.349	-0.295	-0.318	-0.454	-0.305	1

All correlations are significant at 0.001 level.

In Figure 3, standardized coefficients related to second-order CFA, which include JeffSPLL-MS and LLTS measurement models, are given. Based on this structural equation model, a strong Pearson correlation (r=0.624) was obtained between total scores (JEF) obtained from the JeffSPLL-MS and the LLTS measurement models (YBÖÖ) at the level of p<0.001. The relatively high correlation between JeffSPLL-MS and LLTS measurement models showed that the scales were highly interrelated. Considering the fit indices (goodness-of-fit measures) mostly in “good fit” situation (χ2/df=1.29 (<2, Good Fit), RMSEA=0.042 (<0.05, Good Fit), NFI=0.933 (>0.9 & <0.95, Acceptable Fit), RFI=0.903 (>0.9 & <0.95, Acceptable Fit), CFI=0.66 (>0.95, Good Fit), TLI=0.955 (>0.95, Good Fit), the estimated Pearson correlation coefficients are shown in Table 1.

Figure 3

Similar scales validity (standardized coefficients) tested by JeffSPLL-MS and LLTS measurement models tested with second order CFA.

Reliability

In order to determine the internal consistency reliability values of JeffSPLL-MS, LLTS and their subscales, Cronbach α coefficients were calculated and listed in Table 2. When the findings were analyzed, the stratified α coefficients for both JeffSPLL-MS and LLTS were detected to be calculated higher than related Cronbach α values as expected (Table 2). In addition, the test-retest reliability values of the scores obtained at a four-week interval were calculated as being between 0.609 and 0.712 (Table 2).

Discussion

This study is the adaptation of the Jefferson Scale of Physician Lifelong Learning Scale for Medical Students (JeffSPLL-MS) into Turkish. Formal, informal, and non-formal education should be continued intertwined in medical education in order to keep up with the requirements of the today’s world, where information and technology are rapidly changing. The habit of lifelong learning is the most basic element that will sustain this adaptation for physicians after graduation from medical school. In fact, lifelong learning in the field of health is one of the most basic professional competencies that should be acquired during school years, following the rapidly changing professional knowledge[17,19]. The adaptation, and application of JeffSPLL-MS into Turkish will provide useful information about the orientation of the medical students to the LL. It was shown in various studies that the original English version (JeffSPLL-MS) has yielded valid and reliable results[11]. Within the scope of this research, the psychometric properties of the scale were examined.

In a study using JeffSPLL-MS, a three-factor structure emerged in the factor analysis, and these three factors were defined as: “learning beliefs and motivation”, “knowledge seeking skills”, and “taking learning opportunities into account”. It was found that medical students’ lifelong learning skills were developing progressively from the preclinical period to their clinical years, and it was stated that self-directed learning activities were effective in developing lifelong learning skills. Wetzel et al. found the Cronbach’s Alpha coefficient of the lifelong learning scale JeffSPLL-MS as 0.77 in their study[11]. The JSPLL scale was also adapted to Portuguese. Cronbach’s Alpha coefficient 0.89 of that adapted scale was found when applied to physicians, and factor structure was found to be compatible with the original scale with four factors excluding three items. The four subscales determined in that study were conceptually related to the previously defined components of lifelong learning[18]. Adaptation of both JeffSPLL and JeffSPLL-MS to the Chinese language was also done to determine their validity and reliability coefficients. In a study in which JeffSPLL was applied to the primary care physicians, the Cronbach‘s Alpha values were found as 0.86 and 0.77 for the medical students, and evidently the result obtained was similar[20]. The results of the factor analysis showed that the three-factor structure was similar to previous studies, and it was stated that JeffSPLL evaluated the tendency towards lifelong learning of the primary care physicians well. Our Cronbach’s Alpha value of the JeffSPLL-MS lifelong learning scale, which we adapted to Turkish is higher than these results.

When looking at the definitions of reliability and validity, reliability was defined as the degree of consistency and accuracy of a test or scale used for measurement, while validity is known as the degree to which a test or scale measures what is intended to measure or the conformity of the measurement material or the method to the desired feature. Cronbach stated that the higher the Cronbach’s Alpha coefficient, the more consistent the items were with the scale, and the items that examine the similar characteristics[16,17]. As a result, item-total correlation and Cronbach’s Alpha values were sufficiently high or above the expected limits in our findings, so items were not removed from the 14-item draft scale. The high item-total score correlations and Cronbach’s Alpha results in this study showed that the 14-item scale was reliable.

Conclusion

Today, LL skills of medical students are very important due to the rapid increase of medical knowledge and adaptation to technological developments. Some valid and reliable instruments are needed to measure these LL skills. In our study, it was revealed that Turkish version of Jeff-SPLL-MS was suitable in terms of validity and reliability. This scale can be used in research aiming to determine the LL trends of the Turkish students of medicine who provide healthcare services during their internship or after graduation, to reveal the factors affecting these trends and to monitor the effects of LL activities, and to contribute to the quality of medical education. With well-designed longitudinal studies, LL trends of the physicians can be followed after graduation and compared on a regular basis to see what affects these trends by adding diverse variables that have potential influences on lifelong medical education.

Limitations

The results obtained from the study were based only on the students’ self-reports. Number of medical students participated in the study were limited, also participants were only from a single medical school. In addition, since the participation in the study was on a voluntary basis, all of the students in the medical school could not be reached because there were also non-volunteers and absent students at the application time of the scale. Moreover, mostly 2nd and 3rd year students participated in the survey. Students in all grades and from several medical schools should be enrolled in future studies.