Evaluation of Medical Education virtual Program: P3 model.

INTRODUCTION: In e-learning, people get involved in a process and create the content (product) and make it available for virtual learners. The present study was carried out in order to evaluate the first virtual master program in medical education at Shiraz University of Medical Sciences according to P3 Model.
METHODS: This is an evaluation research study with post single group design used to determine how effective this program was. All students 60 who participated more than one year in this virtual program and 21 experts including teachers and directors participated in this evaluation project. Based on the P3 e-learning model, an evaluation tool with 5-point Likert rating scale was designed and applied to collect the descriptive data.
RESULTS: Students reported storyboard and course design as the most desirable element of learning environment (2.30±0.76), but they declared technical support as the less desirable part (1.17±1.23).
CONCLUSION: Presence of such framework in this regard and using it within the format of appropriate tools for evaluation of e-learning in universities and higher education institutes, which present e-learning curricula in the country, may contribute to implementation of the present and future e-learning curricula efficiently and guarantee its implementation in an appropriate way.

Introduction

Garrison views e-learning as online learning facility through network technologies. Elearning has been formed in a knowledge-based community and is the product of fast and expanding developments of modern technologies (1-5).

ELearning possesses special components including ease and speed of updating, storage, retrieval, distribution, and sharing with network-based data, conducting learning process and live communication with the learners via computer and by means of internet standard technology, focus on comprehensive paradigm in learning, educational flexibility and learning, learning redundancy capability and learner- centered learning (6).

Actually, reduction in educational costs is deemed as one of the reasons, which have caused e-learning to be highly noticed today (7).

While developing e-learning products and preparation of opportunities for e-learning has been posited as one of the fields of higher education that is increasingly expanded, our knowledge and awareness about the effectiveness of these new attitudes about learning are restricted because of the shortage of valid scientific evaluations (2).

Anyway, some studies have been so far conducted about evaluation of e-learning, in each of which certain goals and perspectives have been addressed.

In a study conducted to examine and explain a framework for qualitative guarantee of e-learning all over Taiwan, the attitude of participants in virtual educational program was investigated. In this survey, two types of educational programs were explored including E-Learning Service Certification (E-LSE) and E-Learning Course- ware Certification (E-LCC). The analysis signified that the quality of the framework had adequate validity and authenticity and it showed that the participants in both E-LSE and E-LCC curricula felt satisfied with their educational program. The behavioral analysis indicated that in E-LCC method, the positive and growing attitudes were only directed toward improving the acceptance of E-LSC among the participants; also, in E-LCC, positive orientations referred to further preferences and adaptability to improve and facilitate the acceptance and agreeableness of E-LCC (8-11).

The multiplicity of effective factors, variables and diverse classifications, which have been implemented from several perspectives, is one of the paramount challenges in evaluation of e-learning. To assess an e-learning setting, it necessitates identifying the main and effective key factors in implementation of electronic programs successfully including the elements. In this regard, the main success factors may be due to activities and elements which should be confirmed in order to ensure the successful implementation of curricula (2).

Khan’s P3 (People, Process and Product) model assumes several factors as effective in creation of an appropriate e-learning environment. The presented continuum in this model includes eight dimensions as follows: institutional, pedagogical, technological, interface design, management, evaluation, resource support and ethical.

The present study was designed to evaluate the virtual course of medical education in Shiraz University of Medical Sciences based on Khan’s P3 e-learning model to answer the quality of different elements of the course in students and directors’ perspective.

Methods

This evaluation research study with post single group was designed to determine the effectiveness of a virtual program in master degree at Shiraz University of Medical Sciences (SUMS). All students (60) who had participated in this virtual program in more than one year and 21 experts including teachers and directors participated in this evaluation project. According to P3 e-learning model, an evaluation tool (questionnaire) with 5-point Likert rating scale (from 0 “never” to 4 “perfectly exists”) was designed and applied to collect the data. The questionnaire covered 8 dimensions of the model.

Content validity of the questionnaire was checked by 3 professors in e-learning and its total reliability (r=0.92) was confirmed.

Internal consistency of the eight dimensions was measured through Cronbach’s alpha coefficient (Table 1)

Table 1

Alpha coefficient in eight dimensions of the questionnaire

Dimensions	Alpha coefficient	95% confidence interval of the difference
		Lower	Upper
Institutional	0.97	0.75	0.99
Management	0.87
Technological	0.95
Pedagogical	0.87
Interface design	0.97
Resource support	0.97
Evaluation	0.98
Ethical	0.78
Total	0.92

The final questionnaire was emailed to all 110 students who had participated in the virtual education for more than one year; all students 60 filled out the form and returned the email.

The researchers asked all 21 directors and teachers who were involved in the program to fill out the questionnaire and they accepted.

Results

The students’ perspectives on different dimensions and criteria of the virtual course are shown in Table 2.

Table 2

Students’ perspective on e-learning dimensions

Dimension	Criterion	Mean±SD
Institutional and management	Administrative affairs	1.74±1.52
	Educational affairs
	Service affairs
	Delivery and content management
Technological	Infrastructure	1.51±1.17
	Hardware
	Software
Pedagogical	Content	2.05±1.13
	Goal
	Media
	Design
	Organizing
	Learning strategies
Interface design	Design of page & site	2.30±1.90
	Content design
	Guidance
	Accessibility
	Ease of usability
Resource support	Online support	1.17±1.17
	Online sources
	Offline sources
Evaluation	Curriculum evaluation	1.80±1.11
	Learners assessment
Ethical	Diversity of learners	2.02±1.25
	Legal affairs

Teachers’ perspectives on different dimensions and criteria of the virtual course are shown in Table 3.

Table 3

Teachers and directors’ perspectives on elearning dimensions

Dimension	Criterion	Mean±SD
Institutional	Administrative affairs	2.36±0,66
	Educational affairs
Management	Content production management	2.57±0.83
	Content presentation management
Technological	Infrastructure	3.09±0.81
	Hardware
	Software
Pedagogical	Goal	2.77±0.70
	Media
	Design
	Organizing
	Learning strategies
	Evaluation of learners
Interface design	Content design	2.46±0.84
	Guidance
	Accessibility
Ethical	Disciplines and regulations	2.18±0.98
	Legal affairs

Discussion

Based on the findings from the students, resource support of the virtual program was in inappropriate condition in both online and offline sources (with means of 0.73 and 0.77). Similarly, it can be inferred from the results that e-learning in Shiraz University of Medical Sciences is not at a favorable level in all dimensions.

According to the teachers and directors’ views, technological domain with mean 3.18, especially in hardware and software criterion was in a better condition than other domains. However, this superiority is also relatively shown in subordinated parts of other domains as well (including accessibility and guidance in interface design). Generally, they evaluated this virtual course at an average level. The best condition was related to technological dimension while the worst status was related to the ethical dimension (12). The results revealed that elearning in SUMS is far from the favorable status.

Conclusion

The best advantages of applying Khan’s P3 model was related to its comprehensiveness that includes all effective cornerstones in e-learning system and also the other one is that Khan’s continuum was based on features and requirements of learning environment. Thus, presence of such framework in this regard and using it within the format of appropriate tools for evaluation of e-learning in universities and higher education institutes, which present e-learning curricula in the country, may contribute to implementation of the present and future e-learning curricula efficiently and guarantee its implementation in an appropriate way.