Health science students' attitude towards research training programs in the Kingdom of Saudi Arabia: Reliability and validity of the questionnaire instrument.

OBJECTIVE: The aim of this paper was to determine the validity and reliability of a questionnaire tool for measuring students' attitudes toward components relevant to research training programs in the Kingdom of Saudi Arabia.
MATERIALS AND METHODS: The paper reports the responses of 564 Saudi Arabian students from seven government universities to the questionnaire comprising 16 items on 3 conceptual subscales, which measured students' attitude to research activities in the college; students' opinion of faculty involvement in research; and infrastructural facilities in the college. The results of this study provide the final scale, with all the 16 items of the initial Likert scale, for which strong evidence was obtained.
RESULTS: Results indicated that the students' attitude toward the research (SAR) scale had three latent factors, which explained 62% of the variance: The three subscales measured includes: (i) Research activities offered in the college, (ii) students' opinion of faculty involvement in research, and (iii) infrastructural facilities offered in the college for research. The full scale including three subscales had good internal consistency (r s = 0.72 and α = 0.77 for full scale; and α between 0.71 and 0.79 for three subscales).
CONCLUSION: This study provides evidence of reliability and validity of the SAR scale for the measurement of students' attitudes toward research training programs in Saudi Arabian Universities. The research findings will provide the basis for further research on health science students.

INTRODUCTION

The National Commission for Academic Accreditation and Assessment (NCAAA) is responsible for the academic accreditation of Universities in the Kingdom of Saudi Arabia (KSA). NCAAA has instituted several key performance indicators to measure various attributes of the quality of higher education institutions. One of the requirements of this is the evaluation of students attitudes toward research training programs. There has also been a significant shift toward providing health science students with early research experience in their curriculum.[123] The resident research activities have several benefits such as the teaching of skills relevant to evidence-based medicine, promotion of lifelong learning, and an inspiration to residents' career choices and continued scholarly work.[45]

Previous studies which also showed the relationship between research programs and medical education indicated that research had a positive impact on the motivation of medical students.[6789] Similarly, it is likely that doctors who had participated in a scientific program during their academic career would make more accurate diagnoses and better professional decisions.[10] Despite these merits of research work, many obstacles lie in the path of health science students who want to take part in research. This study was thus conducted to explore the attitude of Saudi Health Sciences toward their participation in research. It was expected that, any issues raised by the students in the course of the survey of their attitudes can be easily addressed. Appropriate reforms can be undertaken to improve the quality of research training programs in KSA.

Moreover, at the undergraduate level, students tend to have a negative view of courses on research methods. A better understanding of these attitudes is, therefore, necessary to help instructors create a more positive attitude in their students toward the courses in order to facilitate their learning of research.[11] Exploration of the literature, revealed that instruments for measuring the attributes of research training programs from the students' point of view were limited. Through this study, a standard questionnaire instrument tool was devised to garner the research attitudes of the students, and establish its appropriate validity and reliability.

The primary aim of this study was to develop a valid reliable instrument to measure students' attitude toward research training programs. It was to discover the opinion of students about the availability of Research Training Programs in selected Health Sciences Colleges in KSA. Furthermore, it dealt with the degree of involvement of the students in these programs and the obstacles they encountered in their attempt to get involved in research.

MATERIALS AND METHODS

The students' attitude toward the research (SAR) questionnaire was used to assess the attitude of students of the health sciences toward research training activities offered at the level of higher education institutions in KSA. It was structured in a fashion that would allow the results to be interpreted in the light of national and international recommendations. This instrument consisted of 16 items related to three different attitude subscales, as follows: (a) Attitude of students to the research activities offered in the college, (b) students' opinion of faculty involvement in research, and (c) students' opinion of the infrastructural facilities offered by the college for research. Each item of the instrument used a 5-point Likert scale ranging from 1-strongly disagree to 5-strongly agree.

A total of 564 students were invited by stratified sampling method from 19 different health science colleges from seven Government Universities in KSA. Care had been taken to get a wide representation of samples by covering all provinces of KSA.

The study was conducted in a 12-month period covering the academic year 2011-2012. The students belonging to the health sciences colleges cluster of the seven selected government universities located in four different geographical zones in the KSA were the focus of this study. Only the final year/internship students in the selected health sciences colleges were recruited as the sample for this study. It was based on survey data collected as part of the study on students' attitudes toward research training program. A self-administered questionnaire consisting of 16 closed-ended items with ordinal-level response choices was used to assess attitudes of university students toward research training programs.

The test of validity and reliability of the questionnaire was based on the undergraduate students' opinion on the research training program offered in the selected health sciences colleges in KSA.

The evaluation of questionnaire's reliability-internal consistency was made possible by Cronbach's α,[12] which is considered the most important reliability index, and is based on the number of the variables/items of the questionnaire, as well as on correlations between the variables.[13] The reliability of the instrument means that its results are characterized by repetitiveness and not connected with measurement errors.[14] Such reliability is evaluated by Cronbach's α coefficient.

An exploratory factor analysis was conducted to assess the dimensions of the students' attitude to the research training program scale. The author used principal axis analysis in SPSS version 19.0 (Illinois, USA) as the extraction method. Based on the relatively high correlations among the majority of items (>0.3), an oblique rotation was performed using Promax. Three criteria from Green and Salkind were applied to determine the number of factors to be retained: (1) The absolute values of the Eigenvalues; (2) the relative values of the Eigenvalues; and (3) the relative interpretability of the rotated solutions. In addition, a scree plot and variance explained by the factor solution were also considered in making decisions to retain or exclude factors. Then a principal components analysis with Varimax rotation which produces the dimension of differentiation was used to confirm the scale construct validity. To determine if the sub-scales were suitable for factor analysis, two statistical tests were used. The first was the Bartlett's test of sphericity, which examined the inter-independence of the subscales of the scale, and the second was the criterion Kaiser-Meyer-Olkin (KMO) (KMO Measure of Sampling Adequacy, KMO),[15] which examined sample sufficiency. The main method of extracting factors is the analysis of main components with right-angled rotation of Varimax type, so that the variance between loads of variables are maximized on a specific factor, thus reducing the size of small loads, making big loads bigger, and finally, minimizing those with intermediate values.[16] This means that the factors (components) extracted were linearly irrelevant.[17] The criterion of Eigenvalue or characteristic root (Eigenvalue) ≥1 was used to define the number of factors that were kept.[18] Model acceptance was based on two criteria: (a) In order to be included in the variable cluster of a factor, each variable must have loaded to it >0.5 and (b) <0.4 to the rest of the factors.[19] Moreover, each factor must have more than two variables. In addition, it was considered that on the basis of common variable communalities, the variables with high communality (h2) imply great contribution to the factorial model.[16]

RESULTS

Reliability statistics [Table 1] showed the value of the coefficient of Cronbach for the research scale as 0.768. This was over the 70%, which is a good value for the internal consequence of the conceptual construction of the investigated scale.[20] If release of units continues in the same way, i.e. with the standardized value of the variables, the coefficient Cronbach α will slightly increase the value of α to 0.788. This means that even if the number of the items is increased, Cronbach α will take the value of 0.788.

Table 1

Reliability statistics on SAR scale

The table scale statistics [Table 2] gives scores that are related to the entire scale, which presents a mean of 49.88 and a standard deviation of the class of 6.822 units. Table 3 shows the detailed results of the evaluation of the internal consistency for each sub-scale (factor) of the questionnaire of SAR, and also indicates that the internal consistency of the questions making up each sub-scale was extremely high (Cronbach's α >0.7). The integrity of the instrument as a whole has been demonstrated by invariably positive and significant inter-factor correlations [Table 4].

Table 2

Scale statistics

Table 3

Evaluation of the internal consistency of the sub-scales of questionnaire for evaluation of the SAR (Cronbach's α)

Table 4

Correlation between the sub-scales

Initially, the relevance of the data used for the factor analysis was confirmed. The statistical criteria KMO (KMO = 0.524) and the Bartlett test of sphericity (value 556.442, P < 0.001) [Table 5], indicated that the raw data were suitable for the implementation of factor analysis. Eventually, from the following values [Table 6] of common communality, we learnt that the majority of the questions had a value higher than 0.50, which indicates that the quality of the measurements is satisfactory.

Table 5

KMO and Bartlett's test

Table 6

Communalities on SAR scale

Factor analysis was performed for the evaluation of the construct validity of the questionnaire for the SAR. From the 16 original questions used in the factor analysis, three factors came off, using the Kaiser criterion and Varimax orthogonal rotation. The overall rate of variation of the initial data indicated that three factors interpreted in common, amounted to 62.079%. Factor analysis extracted three factors which conjointly explained 62.079% of the variance in students' attitudes [Table 7].

Table 7

The results of factor analysis on SAR scale

DISCUSSION

The present study has achieved its main objective, namely the validation of the SAR questionnaire. The initial face validity and content validity were confirmed by the construct validity generated from the factor analysis. The internal consistency and reliability of the extracted factors were ascertained by Cronbach's α coefficients. The integrity of the instrument as a whole has been demonstrated by invariably positive and significant inter-factor correlations.

The 16 items making up our instrument is intermediate among the reported range of 13[21] to 100.[22] The number of three extracted factors is modal among the reported range of 3[23] to 11[24] factors. The explained variance of 62.079% is exceeded by only one out of 13 studies ranging in variance from 44%[25] to 68%.[26] Our within-factor α coefficients ranging from 0.71 to 0.79 were intermediate among 25 other studies in which the range reported was from 0.43[27] to 0.90.[28] These comparisons justify the recommendation of the use of this questionnaire on students' attitude toward research training programs in various academic settings. The recommendations are as follows: (1) The results of this study need to be re-examined with other samples of students studying different courses. (2) Relationships between students' attitude and achievement in research have to be explored.

Overall, the study results validate the utilization of the SAR scale in measuring students' attitude toward research. Further studies are needed to identify and incorporate some hitherto unused determinants of students' attitude.

CONCLUSION

The study successfully developed and validated a students' attitude toward research training program questionnaire suitable for health science colleges. The following three attributes make SAR strongly commendable for the investigation of the state of SAR in various academic settings. They are the (i) Explained variance of 62.079%, (ii) the overall 0.768 α coefficients of internal consistency reliability, and (iii) the invariably positive and significant inter-factor correlations. Moreover, sample sufficiency tested by KMO, and content and construct validity of the instrument tested by product moment correlation and principle component analysis, indicated a high reliability (internal consistency) and good content, construct and criterion-related validity. This is, therefore, a reliable valid tool for measuring students' attitude toward research training program in health science colleges. It is therefore, recommended that this tool be incorporated or adapted by colleges and universities as one of the means of assessing SAR training programs.