Evaluation of the effectiveness of educational medical informatics tutorial on improving pharmacy students' knowledge and skills about the clinical problem-solving process.

Objective: To investigate the effectiveness of an online tutorial and its impact on improving knowledge and skills of pharmacy students in the clinical problem-solving process that is necessary to implement pharmaceutical care.
Methods: This is a prospective interventional study conducted during the COVID-19 pandemic restrictions using four novel templates. The first two levels of Kirkpatrick's Model (Reaction and Learning) were used.
Results: 129 participants completed all of the online training parts. The findings indicated a significant improvement in the students' knowledge and skills. The participants achieved higher score following the tutorial than the baseline, with a statistically significant difference (p < 0.001). There was a significant improvement in the number of detected treatment-related problems. The majority of students were satisfied with the overall training process and stated a high evaluation score out of 10 (mean = 7.93 ± 1.42, median = 8.00).
Conclusion: The educational intervention achieved a substantial positive impact on decision-making skills of participating students and was considered effective in helping them attain basic skills such as teamwork, peer assessment, communication and critical evaluation. Healthcare providers must work together to ensure accurate medication use during care transitions. Pharmacists, as medication experts, play an important role in the implementation process. Pharmacy educators must prepare pharmacy student to use pharmaceutical care in their future practice. Copyright: © Pharmacy Practice.

INTRODUCTION

Over years, the role of pharmacists has expanded from traditional dispensing to patient-focused pharmaceutical care (PC) practice.1,2 Pharmacists are now working collaboratively with other healthcare practitioners as a neighborhood of the health-care team to provide optimized patient care services in hospitals and various clinical settings.3-7 As pharmacists are now required to possess closer contact with patients,8 the general public naturally expects them to be able to handle challenges competently and in the best interests of their patients.9

The term PC was first defined by Hepler and Strand as “the responsible provision of drug therapy for the purpose of achieving definite outcomes that improve a patient’s quality of life”.10 More recently, PC was defined by the Pharmaceutical Care Network Europe as “the pharmacist’s contribution to care of individuals in order to optimize medicines use and improve health outcomes”.11

Many studies have determined that involvement of pharmacists towards PC services improves patients’ health outcomes through the identification and prevention of TRPs in several diseases12-19 and reduces health costs.20-23

However, several barriers have impeded the implementation of PC practice worldwide. These include lack of pharmacists` time, poor clinical knowledge and communication skills, insufficient pharmacists’ self-confidence in addition to the negative attitudes of pharmacists themselves toward performing PC.24-29

International pharmacy bodies like The Accreditation Council for Pharmacy Education (ACPE)30 and The Centre for the Advancement of Pharmacy Education (CAPE)31 require trainee (and practicing) pharmacists to possess an appropriate awareness and competencies within a patient-centred model of practice to enable evidence-based decision-making.31,32 This needs academically well-designed teaching components embedded within pharmacy curricula around evidence-based decision-making.

Accordingly, it’s vital to organize pharmacy practitioners of today and tomorrow. Different studies have described the role that the educators of pharmacy school should play in supporting the PC practitioners.33-38 They have to design educational tools that promote the performance of pharmacists who are willing to implement PC in their practice and to ensure that students have a deep understanding of the principles and practices of PC.39 In addition, they ought to establish in student’s appropriate clinical knowledge, communication and problem-solving skills, and self-confidence to take the responsibility for providing PC.39 One of the challenges faced by pharmacy educators is the motivation of students to provide PC.40 However, enhancing students’ tendency toward PC will be reflected on pharmacy educators readiness to increase PC’s supply in practice.40 This combination of knowledge, skills and attitudes is required for widespread acceptance and implementation of the concept of PC.

There is a recognized value of PC and clinical problem-solving as a process that reduces medication errors and patient harm.41,42 However, health care providers generally, pharmacists are not an exception, receive little formal education during their college years on how to collect patient`s data, assess these data including the medications, design and implement the patient`s care plan, educate the patient, with appropriate follow-up.41,42 This may be attributed to the lack of awareness and unclear understanding of who has the responsibility for providing this service.43,44 Therefore, this study aims to investigate the effectiveness of a tutorial training and its impact on improving knowledge and skills of pharmacy students in the clinical problem-solving process that is necessary to implement PC.

METHODS

Study setting and participants

This is a prospective interventional study that was conducted from October 2020 to February 2021 at the faculty of pharmacy at Applied Science Private University (ASU), a well-ranked Jordanian university located in Amman, Jordan. Pharmacy students enrolled in the Clinical Pharmacy and Therapeutics Lab module were invited to participate in the study during the study period.

This interventional trial was undertaken in the midst of the COVID-19 outbreak. As a result, researchers were forced to change the course and give it online via Microsoft Teams® rather than using the intended face-to-face delivery method.

The ethics committee of ASU granted approval under the number 2021-PHA-4-5. All students were advised that participation was voluntary and that their responses would be kept completely confidential.

Designing and implementing the tutorial intervention

We developed the educational intervention in light of the available literature and the researchers’ personal experiences. Prior to initiating the intervention, the material was reviewed and confirmed to ensure the information supplied was clear and accurate. Additionally, after developing the tutorial, it was pilot tested on 19 students during the preceding semester (not included in the analysis). Additionally, the material was re-evaluated following the pilot research.

The initial part of this study involved pre-testing students prior to delivering the educational intervention. The second part was the educational intervention’s delivery. The third part involved administering the post-intervention test. Finally, the fourth part comprised a focus group discussion with the intervention’s participants to get comments about the intervention. Three researchers (all with PhD, degree in clinical pharmacy and a minimum of seven years’ experience in the clinical field) oversaw the educational intervention and data collection. Four innovative templates were used before, during, and following this instructional course: data collection (Appendix-Table A), basic calculations (Appendix -Table B), medication assessment (Appendix -Table C), and care plan (Appendix -Table D). Different versions of the tables in Appendix – Tables C and D were utilized depending on the patient’s medication and disease status. These templates are part of a larger project coordinated by the first author that includes a website and a mobile application aimed at providing PC support (https://www.asami-draacare.com).

Table A

Data Collection

1.	Demographic Data:
•	Who is the patient?
•	Age:
•	Gender:
•	Education and Occupation:
2.	Current Issues
⁘	Chief Complain:
⁘	History of Present Illness:
•	Location:
•	Characteristic of symptoms:
•	Timing (onset, duration, frequency):
•	Severity:
•	Factors:
•	Aggravating factors:
•	Alleviating factors:
•	Environment:
•	Other symptoms:
•	Differential diagnosis:
⁘	Review of Systems:
⁘	Physical Examination:
3.	Medical problems
•	Past, acute and chronic diseases:
•	Stage / type / class:
•	Current status:
•	Duration:
4.	Patient History
•	Surgeries:
•	Hospitalization:
•	Vaccination:
•	Allergies:
•	Family history:
•	Social history:
•
5.	Treatments
•	Drug Details:
•	Duration:
•	Time of administration:
•	Adherence:
6.	Tests
•	Vital signs:
•	Lab tests:
•	Other tests:
7.	Special Situation
•	Pediatric, geriatric, pregnancy, breastfeeding, ethnicity.
•	Limitations (NPO, bedridden, inability to swallow)
•	Renal impairment
•	Hepatic impairment
•	Allergies
•	Sensitive issues / barrier to communication
•	Abuse

Table B

Basic calculations

Date
Weight (kg)
Height (cm)
https://www.feettometres.com/
BMI
[Weight (kg) / Height (m)^2]
https://www.nhlbi.nih.gov/health/educational/lose_wt/BMI/bmi-m.htm
BMI Categories:
Underweight = <18.5
Normal weight = 18.5–24.9
Overweight = 25–29.9
Obesity = BMI of 30 or greate r
IBW
https://www.mdcalc.com/ideal-body-weight-adjusted-body-weight
This formula is only an approximation, and is generally only applicable for people ≥152.4 cm tall. For patients under 152.4 cm, one commonly-used modification is to subtract 1 kg for each 2.54 cm below 152.4 cm.
Renal impairement (eGFR)
https://www.kidney.org/professionals/kdoqi/gfr_calculator
eGFR: 90–120 mL/minute/1.73 m2

Table C

Assessment of medications

Date
Drug name (scientific and trade name) / strength / route / frequency
Medical problem or health care nee d
Goals of treatment
Category
PTA, current, or discharge;
1.	P, OTC, or CAM
2.	PRN; Stat; or regular
3.	Start date
Stop date
Time
Indication
Effectiveness
Safety
Dosing regimen
Product preparation (procedures, methods), compatibility and stability
Drug interactions (drugs, foods, tests)
Patient (knowledge, adherence)
Reference

Abbreviations: PTA: prior to admission; P: Prescribed; OTC: over-the counter; CAM: complementary and alternative medicine; PRN: per registered nurse (as needed).

Table D

Patient Care Plan

Date
Medical problem or healthcare need
Goal(s) of Treatment
Treatment related problem (TRP)
[Indication, Effectiveness, Safety, Patient, Others]
Intervention(s) / Recommendation(s)
Follow-up (Monitoring and Evaluation)
Monitoring parameter(s) (therapeutic or toxic) / Endpoints / Frequency
Follow-up (timeframe)
References

All surveys were distributed in English, the official language of pharmacy education in Jordanian universities. To ensure face validity, three academics with considerable experience conducting clinical studies and a broad spectrum of professional clinical experience examined the questionnaires prior to the current investigation.

Part 1: Pre- intervention part

After acquiring students’ informed written agreement, a pre-training data collecting form and ability to analyze a clinical case was used to assess students’ knowledge and skills regarding the clinical problem-solving process in less than two hours. Students were requested to complete a questionnaire throughout this time period in order to obtain data on demographic variables.

Part 2: The delivery of the intervention

After one week, the training instruction began and lasted four weeks (a total of 10 training hours). Throughout the week, tutorials were offered and included problem-based learning strategies as well as appropriate hands-on and small group activities.

This section of the study introduced students to the clinical problem-solving process. This section is divided into three phases, as follows:

Phase I

Through didactic lectures, this phase was aimed to familiarize students with the clinical problem-solving process. This phase involved delivering a series of didactic lectures via Microsoft Teams® on pharmaceutical and disease information resources. The following websites were discussed and used throughout the course: accesspharmacy.mhmedical.com, https://online.epocrates.com, www.Drugs.com, www.elm.jo and www.medscape.com. To get the essential information, participants must consult at least two of these sources. Additionally, during this time period, pharmacy students were taught to decision-making and professional communication abilities.

Phase II

Three skills-based workshops were performed, also via Microsoft Teams®, to facilitate the translation of theory into practice. Workshop participants were exposed to simulated patient case studies and participated in group discussions. The cases involved a variety of various body systems, including cardiovascular, pulmonary, rheumatic, endocrine, and infectious illnesses. Following these cases, the researchers moderated a structured and open discussion. The purpose of these workshops was to help participants transition from theoretical knowledge to application and active decision-making.

The lecturer for this course (a ground researcher) interacted with students via video conference, providing relevant examples, discussions, workshops, and motivational feedback.

Phase III

This part evaluated the competencies acquired by attendance at lectures and workshops on the subject. Students were divided into groups and assigned cases. Each group was assigned a portion of the case study to deliver in a presentation manner.

Part 3: Post-intervention

Following that, cases were discussed and skills in data collection, assessment, care plans and patient education were evaluated over a 5-week period, with each group having two hours of discussion. Then, students’ knowledge and skills about the clinical problem-solving process were re-evaluated, as well as their level of satisfaction with the training workshops. Additionally, we compared the final exam results and presentation and case discussion skills of students enrolled in the current interventional tutorial to those who got the standard tutorial the previous semester in the same setting.

Students were evaluated in this study using an oral test, multiple-choice and written exams and tutorial participation. Three lecturers (all with a PhD. in Clinical Pharmacy) and three pharmacists facilitated the tutorial.

An online post-intervention test was administered. Kirkpatrick’s Model’s first two levels (Reaction and Learning) were utilized to evaluate the new tutorial tool’s efficacy. The Kirkpatrick model can be used before, during, and following training to demonstrate the training’s worth to an organization or an individual. The model is divided into four levels.45 Due to the cost and difficulty of assessing levels 3 and 4 (Behavior and Results, respectively), only the first two levels (Reaction and Learning) were tested during this course.

Level 1 (Reaction; “to what extent participants react positively to the learning event”) was measured using a satisfaction questionnaire that judged pharmacy students’ positive reactions to the tutorial. To ensure the questionnaire’s clarity and usability, it was face and content validated by the study’s researchers. Each statement was evaluated using a five-point Likert scale (strongly agree, agree, neutral, disagree, and strongly disagree).

Two criteria were used to assess Level 2 (Learning; “the extent to which participants acquire the desired knowledge, abilities, and attitudes as a result of their involvement in the learning event”):

1) Acquired knowledge of the clinical problem-solving procedure by students. This section included an exam designed to assess pharmacy students’ knowledge of the clinical problem-solving process. Three researchers face and content validated the exam in order to improve the clarity and comprehensibility of the questions. The questionnaire had 30 multiple-choice questions covering various topics such as data collection, assessment, treatment plan development, and patient education.

2) Acquired abilities of students to apply the clinical problem-solving method. During this phase, students were divided into groups of five to seven and asked to analyze a variety of clinical cases of similar complexity.

Part 4: Focus group discussions

Additionally, focus group discussions were held online via Microsoft Teams®. We used a standardized interview protocol. Students who completed all three phases were invited to engage in an online focus group discussion. This phase of the study aims to collect comments from participants regarding the intervention trial. It provided an opportunity for reflection on the intervention and its impact. The ground researcher transcribed, de-identified and translated this conversation from Arabic to English. The ground researcher coded, conceptually analyzed and highlighted emerging topics from the translated discussion. The themes were validated following a separate analysis of the data by the study team (three researchers, all with Ph.D. in clinical pharmacy).

Data analysis

Data was analysed by the Statistical Package for Social Science (SPSS Inc., Chicago, IL, USA) version 24. Mean ± standard deviation (SD) was used to express the continuous variables and frequencies (percentages) were used to express categorical variables.

Checking for normality was carried out using Shapiro-Wilk test (with p>0.05 indicating a normally distributed continuous variable) or by visual inspection of a Normal Q-Q Plot. Paired t-test was used to evaluate pre-post changes in the knowledge and skills scores; if the data was not normally distributed, Wilcoxon sign rank test was used instead. Independent t-test was used to compare the scores between the students enrolled in the current interventional tutorial and other students who received the traditional tutorial in the preceding semester. For all statistical analysis, p < 0.05 was considered statistically significant.

RESULTS

Among the 155 students who registered to in the Clinical Pharmacy and Therapeutics Lab module, 129 participated in this study and completed the educational training (response rate 83.2%), with a mean age (± SD) of 23.0 (±1.9) years. Female represented 74.4% of the study sample (n=96). Regarding year level, the majority (n=126, 97.7%) of the participants were in their fifth (final) year of the undergraduate pharmacy program. Whilst, 2.3 % of the participants were fourth year students who had completed other prerequisite modules. None of the participating students was repeating this module due to a previous failure. As stated by the students, around half of the students (53.5%) had attended a course/workshop about the clinical problem-solving process (Table 1).

Table 1

Demographic data of the study participants (N = 129)

Parameters	n (%)	Mean (SD)
Gender
Male	33 (25.6)
Female	96 (74.4)
Age (years)		23.00 (±1.90)
Year of Study
Fourth year	3 (2.3)
Fifth year	126 (97.7)
Attended a course/workshop about clinical problem-solving process
Yes	69 (53.5)
No	60 (46.5)

As shown in Table 2, the majority of students (77.5%) preferred to solve the clinical case as a group rather than individually. Most of the students believed that working as a group improves their skills in problem solving with a mean (± SD) of 4.07/5 (±1.11). Also, most of students agreed that working as a group improved their skills in discussion and decision making with means (± SD) of 4.17 (± 0.99) and 4.16/5 (± 0.91), respectively. In the context of the obstacles that the students could face when they deal with each other as a group, the highest proportion of the students (28.7 %) thought that the time coordination within the group is the most faced obstacle. However, the second highest proportion of the students (25.6 %) did not find any obstacles when dealing with their group.

Table 2

Description of the advantages and obstacles of the group distribution method used during the training course (N = 129)

Parameters	n (%)
Prefer to solve the case individually	29 (22.5)
Prefer to solve the case as a group	100 (77.5)
Working as a group improves your skills in problem-solving
Strongly agree (5)	54 (41.9)
Agree (4)	51 (39.5)
Neutral (3)	11 (8.5)
Disagree (2)	5 (3.9)
Strongly disagree (1)	8 (6.2)
Mean (SD)	4.07 (1.11)
Median	4.00
Working as a group improves your skills in discussion
Strongly agree (5)	55 (42.6)
Agree (4)	56 (43.4)
Neutral (3)	9 (7.0)
Disagree (2)	3 (2.3)
Strongly disagree (1)	6 (4.7)
Mean (SD)	4.17 (0.99)
Median	4.00
Working as a group improves your skills in decision making
Strongly agree (5)	52 (40.3)
Agree (4)	54 (41.9)
Neutral (3)	17 (13.2)
Disagree (2)	3 (2.3)
Strongly disagree (1)	3 (2.3)
Mean (SD)	4.16 (0.91)
Median	4.00
The most obstacle you face when dealing with your group
Time coordination	37 (28.7)
Communication skills and ways	13 (10.1)
Conflicts	4 (3.1)
Convincing others	2 (1.6)
All the mentioned	22 (17.1)
Others	18 (14.0)
No obstacle was found	33 (25.6)

The improvement in students’ knowledge and skills following the intervention was assessed and presented in Table 3 and the results showed that there was a significant improvement in the students’ knowledge and skills.

Table 3

Different aspects of participants` evaluation during the study (N = 129)

Score type	Pre-training	Post-training	The mean difference	95% CI of the Difference		t	df	P value
	Mean (SD)			Lower	Upper
A paired-samples t-test
Case discussion and report score	9.38 (5.71)	14.77 (2.60)	5.39 (3.27)	4.53	6.24	12.458	128	<0.001
Students` self-score
Data Collection	53.07 (21.58)	81.00 (13.87)	27.93 (18.63)	24.68	31.18	17.026	128	<0.001
Assessment	48.56 (24.57)	79.75 (13.60)	31.19 (22.08)	27.35	35.04	16.049	128	<0.001
Care Plan	41.412 (25.10)	78.80 (15.23)	37.39 (23.07)	33.37	41.41	18.405	128	<0.001
Patient Education	57.98 (20.80)	82.98 (13.81)	24.99 (18.02)	21.85	28.13	15.750	128	<0.001
Follow-Up	51.18 (25.28)	79.05 (16.35)	27.88 (21.19)	24.19	31.57	14.944	128	<0.001
An independent-samples t-test
	Traditional tutorial	Interventional tutorial
Final exam score	25.54 (7.46)	43.05 (4.25)	17.51 (6.01)	-19.36	-15.66	-21.43	203	< 0.001
Case discussion and presentation evaluation score	16.16 (4.11)	16.72 (2.54)	0.56 (3.11)	-1.47	0.36	-1.21	203	0.23

A paired-samples t-test was used to determine whether there was a statistically significant mean difference between the students` exam scores (case discussion and report) after the tutorial compared to those scores before the tutorial. As assessed by inspection of a boxplot, there were no outliers in the data. The differences between the pre- and post-tutorial scores were normally distributed, as assessed by visual inspection of a Normal Q-Q Plot. The participants achieved higher score following the tutorial (14.77 ± 2.60) than the baseline (9.38 ± 5.71), with a statistically significant difference of 5.39 (95% CI, 4.53 to 6.24), t (128) = 12.46, p < 0.001 (Table 3).

A paired-samples t-test was used to determine whether there was a statistically significant mean difference between the students` self-reported scores of five skills; data collection, assessment, care plan, patient education, follow-up after the tutorial compared to those scores before the tutorial. There were no outliers in the data. The differences between the pre- and post-tutorial scores of the five parts were normally distributed. As shown in Table 3, all the evaluated five skills were improved significantly following the tutorial, with the highest improvement were reported for the care plan, followed by assessment, data collection, follow-up and finally patient education (p < 0.005).

An independent-samples t-test was run to determine if there were differences in the final exam score (out of 50) between the students enrolled in the interventional tutorial (n = 129) and other students who received the traditional tutorial in the preceding semester (n = 76). The scores for each level of the groups were normally distributed, as assessed by Shapiro-Wilk’s test (p > 0.05), and there was homogeneity of variances, as assessed by Levene’s test for equality of variances (p = 0.160). The intervention group achieved higher score (43.05± 4.25) than the control group (25.54 ± 7.46), a statistically significant difference of 17.51 (95% CI, --19.36 to -15.66), t (203) = -21.43, p < 0.001 (Table 3).

An independent-samples t-test was run to determine if there were differences in the case discussion and presentation evaluation score (out of 20) between the students enrolled in the interventional tutorial (n = 129) and other students who received the traditional tutorial in the preceding semester (n = 76). The intervention group achieved higher score (16.72 ± 2.54) than the control group (16.16 ± 4.11), a statistically insignificant difference of 0.56 (95% CI, -1.47 to 0.36), t (203) = -1.21, p = 0.23 (Table 3).

Table 4 represents the number of treatment related problems detected by students pre-training (TRP-1), and post-training using two approaches; TRP-2 using the skills learned during the tutorial (indication, effectiveness, safety, patient, miscellaneous) or TRP-3 using the previous validated checklist.46 Based on the means comparison, both post-tutorial approaches showed increase in the number of TRPs detected by students with means ± SD of 5.80 ± 5.11 and 5.53 ± 6.48, respectively, compared to pre-tutorial 4.51± 3.03. The paired difference in TRP-3 was insignificant compared to TRP-1. However, there was a significant increase in the number of TRPs detected by students in TRP-2 (1.29) (95% CI, 0.55 to 2.03), t (127) = 3.46, p < 0.001 (Table 4). The easiness and usefulness of TRP-2 and TRP-3 were assessed. 54.3% of the students thought that TRP-2 is easier than TRP-3. Moreover, most of the students (68.2%) thought that TRP-2 is more useful than TRP-3.

Table 4

Number of treatment related problems detected by students pretraining and post-training using two approaches (N = 129)

	TRP_1	TRP_2	TRP_1	TRP_3
Mean	4.51	5.80	4.51	5.53
SD	3.03	5.11	3.03	6.48
Paired differences	1.29 (4.22)		1.01 (6.64)
95% CI of the differences	0.55 to 2.03		-0.15 to 2.17
t	t (127) = 3.46		t (127) = 1.72
p value	p = 0.001		p = 0.088

The student’s satisfaction with the training tutorial about the clinical problem-solving process (post educational tutorial) was assessed (Table 5). Most of the students demonstrated that the educational training tutorial enhanced their learning as overall with a mean ± SD of 4.22/5 ± 0.75. Furthermore, most of the students found that the patient education instructions and the drug information resource instructions were well administrated (mean= 4.27 ± 0.81 and 4.29 ± 0.74, respectively). Also, the bulk of students found that the patient education part was clear and beneficial with means ± SDs of 4.25 ± 0.80 and 4.25 ± 0.80, respectively. Moreover, the majority of students found that the care plan part was beneficial (mean=4.26 ± 0.80), the data collection part is beneficial as well (mean=4.23 ± 0.78), and the drug information resources skills were beneficial (mean=4.30 ± 0.70). Regarding the exam, most of students thought that it was fair (mean=4.11 ± 0.82), in addition, that the personality, ethnicity and gender did not affect the scores of the exam (mean=4.19 ± 0.97).

Table 5

Students Satisfaction with the educational training tutorial about the clinical problem-solving process (post educational tutorial) (N = 129)

No	Statements	5	4	3	2	1	Mean (SD)	Median (IQR)
Part A:
1	The educational tutorial stimulated my interest in Clinical problem-solving process.	37 (28.7)	75 (58.1)	13 (10.1)	3 (2.3)	1 (0.8)	4.12 (0.74)	4 (0)
2	Methods of Clinical problem-solving process introduced in this educational tutorial are beneficial.	37 (28.7)	66 (51.2)	22 (17.1)	4 (3.1)	0	4.05 (0.76)	4 (0)
3	I enjoyed participating in this education training.	35 (27.1)	56 (43.4)	29 (22.5)	7 (5.4)	2 (1.6)	3.89 (0.92)	4 (1)
4	The educational tutorial was easy to understand.	28 (21.7)	42 (32.6)	34 (26.4)	23 (17.8)	2 (1.6)	3.55 (1.07)	4 (1)
5	The educational tutorial made me understand the concept of Clinical problem-solving process.	45 (34.9)	60 (46.5)	21 (16.3)	3 (2.3)	0	4.14 (0.77)	4 (0)
6	The educational tutorial helped me understand the importance of Clinical problem-solving process.	45 (34.9)	52 (40.3)	30 (23.3)	2 (1.6)	0	4.09 (0.80)	4 (0)
7	The educational tutorial enhanced my learning.	50 (38.8)	60 (46.5)	16 (12.4)	3 (2.3)	0	4.22 (0.75)	4 (0)
8	Wide knowledge area covered.	47 (36.4)	49 (38.0)	29 (22.5)	4 (3.1)	0	4.08 (0.84)	4 (0)
9	Wide range of clinical skills covered.	53 (41.1)	48 (37.2)	25 (19.4)	3 (2.3)	0	4.17 (0.82)	4 (0)
10	The educational tutorial well structured & sequenced.	36 (27.9)	57 (44.2)	30 (23.3)	4 (3.1)	2 (1.6)	3.94 (0.88)	4 (0)
11	Student aware of level of data and skills needed.	46 (35.7)	51 (39.5)	27 (20.9)	4 (3.1)	1 (0.8)	4.06 (0.87)	4 (0)
Part B:
1	DI resources instructions well administered.	55 (42.6)	59 (45.7)	13 (10.1)	1 (0.8)	1 (0.8)	4.29 (0.74)	4 (0)
2	DI resources skills are beneficial.	54 (41.9)	54 (41.9)	9 (7.0)	3 (2.3)	0	4.30 (0.70)	4 (1)
3	Data Collection part instructions well administrated.	41 (31.8)	56 (43.4)	28 (21.7)	4 (3.1)	0	4.04 (0.81)	4 (0)
4	Data Collection part is clear.	42 (32.6)	48 (37.2)	30 (23.3)	8 (6.2)	1 (0.8)	3.95 (0.94)	4 (1)
5	Data Collection part is beneficial.	53 (41.1)	57 (44.2)	15 (11.6)	4 (3.1)	0	4.23 (0.78)	4 (0)
6	Medication’s assessment template instructions well administrated.	43 (33.3)	54 (41.9)	25 (19.4)	5 (3.9)	2 (1.6)	4.02 (0.91)	4 (0)
7	Medication’s assessment part is clear.	42 (32.6)	51 (39.5)	24 (18.6)	10 (7.8)	2 (1.6)	3.94 (0.98)	4 (0)
8	Medication’s assessment part is beneficial.	58 (45.0)	44 (34.1)	22 (17.1)	3 (2.3)	2 (1.6)	4.19 (0.91)	4 (0)
9	The educational training helped me to be able to identify treatment-related problems (TRPs).	54 (41.9)	49 (38.0)	20 (15.5)	3 (2.3)	3 (2.3)	4.15 (0.93)	4 (0)
10	Care plan template instructions well administrated.	43 (33.3)	54 (41.9)	26 (20.2)	3 (2.3)	3 (2.3)	4.02 (0.92)	4 (0)
11	Care plan part is clear.	46 (35.7)	50 (38.8)	22 (17.1)	9 (7.0)	2 (1.6)	4.00 (0.98)	4 (0)
12	Care plan part is beneficial.	57 (44.2)	53 (41.1)	14 (10.9)	5 (3.9)	0	4.26 (0.80)	4 (0)
13	Patient education instructions well administrated.	60 (46.5)	48 (37.2)	17 (13.2)	4 (3.1)	0	4.27 (0.81)	4 (1)
14	Patient education part is clear.	58 (45.0)	48 (37.2)	20 (15.5)	3 (2.3)	0	4.25 (0.80)	4 (1)
15	Patient education part is beneficial.	53 (41.1)	58 (45.0)	15 (11.6)	3 (2.3)	0	4.25 (0.75)	4 (0)
16	It is beneficial to include the skills learned in different courses like therapeutics, OTC, training, clinical biochemistry, DI etc.	56 (43.4)	43 (33.3)	20 (15.5)	8 (6.2)	2 (1.6)	4.11 (0.99)	4 (0)
Part C:
1	Hospital case evaluation was fair.	18 (14.0)	60 (46.5)	32 (24.8)	18 (14.0)	1 (0.8)	3.59 (0.92)	4 (1)
2	Final exam was fair.	47 (36.4)	53 (41.1)	25 (19.4)	4 (3.1)	0	4.11 (0.82)	4 (0)
3	Exams well administered.	38 (29.5)	64 (49.6)	25 (19.4)	2 (1.6)	0	4.07 (0.74)	4 (0)
4	Exams very stressful.	32 (24.8)	34 (26.4)	32 (24.8)	18 (14.0)	13 (10.1)	3.42 (1.28)	4 (1)
5	Exams well structured & sequenced.	38 (29.5)	52 (40.3)	33 (25.6)	5 (3.9)	1 (0.8)	3.94 (0.88)	4 (0)
6	Personality, ethnicity, and gender do not affect scores.	63 (48.8)	36 (27.9)	23 (17.8)	5 (3.9)	2 (1.6)	4.19 (0.97)	4 (0)

5: strongly agree; 4: agree; 3: neutral; 2: disagree; 1: strongly disagree.

Finally, the majority of students were satisfied with the overall training process and stated a high evaluation score out of 10 (mean = 7.93 ± 1.42, median = 8.00) (Figure 1).

Figure 1

The Overall students`evaluation for the educational tutorial tool (10: the best, 0: the worst). Mean = 7.93 ± 1.42, median = 8.00.

In addition to the positive changes observed in the post-surveys, focus-group discussions and students’ opinions appeared to endorse findings of enhanced understanding of clinical problem-solving skills. Discussions were transcribed, translated and analysed by the ground researcher and separately analysed by the research team.

Students identified the need for and the impact of such courses on pharmacy practice in Jordan. They repeatedly described the course as “interesting” and “thought-provoking”, expressing the novelty of this topic to them, and depicted this teaching approach as “beneficial”. Participants in this study commented positively about workshop discussions utilized, explaining that this approach facilitates the transforming of the theoretical part of the intervention to a practical interactive model, allowing students to learn from each other.

Students who participated in this study considered this training methods useful and demonstrated enhanced confidence in handling clinical cases that might be encountered in practice.

Most participants suggested incorporating such a course in an earlier year of study, then slightly increasing the level of intensity and complexity of information each time repeated across each year of the five years of the BPharm curriculum. Therefore, learning materials are disseminated across all the degree stages in order for pharmacists to practice in a less stressful environment with more confidence.

DISCUSSION

Nowadays, the pharmacist’s role has developed dramatically, and it is anticipated that pharmacists will play an important role in providing PC services. Previously, pharmacists have not been directly concerned with gathering a good medication history in most hospitals.47 This may be due to the insufficient information of the process, the lack of awareness, imprecise understanding of who has the obligation for delivering PC services, and shifting responsibilities to other healthcare providers for administering the service.48,49 The lack of knowledge, skills and awareness about the service was among the barriers for implementing PC services. Therefore, this study was designed to solve this gap in learning the clinical problem-solving process at pharmacy school years by the development of an innovative learning tool.

Few studies have been conducted to determine the effect of educational programs on students’ ability to obtain medication histories and to identify and resolve TRPs.50-52 These studies have focused on incorporating an interactive learning exercise using simulation50,51 and the use of a short duration video tutorial 52 to educate students about medication reconciliation and the results of these studies indicated that students’ knowledge and skills about medication reconciliation and identifying discrepancies were improved following the educational intervention.50-52 However, medication reconciliation and finding discrepancies are only one part of PC principle. Accordingly, the aim of this study was to develop, implement and evaluate the effectiveness of another teaching method, an online tutorial, to educate pharmacy students about the clinical problem-solving process and how to retrieve patient data from medical files and developing a patient specific pharmaceutical care plan for the identification, resolution and prevention of TRPs that enable them to practice PC and improve decision making process.

Pharmacy education provided at the level of pharmacy schools is considered as one of the most crucial elements in the development of professional pharmacists.50,53,54 Pharmacy educators have the responsibility to provide students with adequate and comprehensive training and educational tools to ensure their ability to provide the PC service to their patients.50,53,54 Therefore, it is important to supply students with the proper skills in a way that the pharmacy educators guarantee that they obtained appropriate skills during their training and to be able to provide PC correctly.50,53,54

New techniques are getting attention in the field of education, involving the usage of online communication and videos.55,56 Visual teaching tools have been shown to improve students’ comprehension and learning.56-58 However, their effectiveness in pharmacy education is not fully known.56,59,60 Problem-based learning still remains the most innovative medical education method as it is characterized by self-directed learning using simulated real-life scenarios as the learning programme in addition of being student-centered learning.61

The findings of this interventional study demonstrate that this training method significantly improved the total knowledge and skills scores about clinical problem-solving process. The participants achieved higher examiner scores (case discussion and report) following the tutorial compared to those scores before the tutorial, with a statistically significant difference, p < 0.001. Also, by investigating the students` self-reported scores, all the evaluated five skills were improved significantly following the tutorial, with the highest improvement were reported for the care plan, followed by assessment, data collection, follow-up, and finally patient education (p < 0.005). We compared the final exam score between the students enrolled in the interventional tutorial and other students who received the traditional tutorial in the preceding semester. The intervention group achieved higher score than the control group, p < 0.001. Unlike a previous recent study which evaluated the influence of the training module immediately after conducting the workshop,52 we evaluated the effectiveness after several weeks which ensure measuring the long-term effect of this educational tutorial.

The study’s findings were consistent with those of prior research done in the United Kingdom and New Zealand in certain aspects.56,62 The researchers in these studies discovered that visual techniques of instruction produced positive effects.56 For instance, videos demonstrating several practical skills and visual aids represented in Magnetic Resonance Imaging (MRI) images were effective and improved the understanding of students about certain pharmaceutical concepts.62 Moreover, enhancement of innovative and interactive learning experience of students were observed.56,62

In terms of students’ ability to detect TRPs, students were able to identify more TRPs following the instruction. As a result, it may be assumed that students who participated in this study were able to put their newly acquired information and abilities to use.

When students’ perceived satisfaction with their training strategy was evaluated, the majority expressed pleasure with the whole training process and stated that the educational tool helped them acquire the notion of clinical problem-solving. This results was consistent with that of Zhang et al., who demonstrated that using videos in the learning process resulted in a higher level of student satisfaction than other techniques.63 Thus, it can be advised and encouraged that visual educational videos be incorporated into the learning process. Additionally, the utilization of online tutorials as a teaching approach benefited from the fact that they were available at any time. In contrast to typical lectures, students can listen to the information multiple times and refer to the recordings if they missed anything.

Upon using sound methods of educational delivery, educational intervention trials have demonstrated an enhancement of the pharmacy practice skills.64,65 Moreover, implementing such pedagogically-framed training has shown an improvement in the quality of patient-care in addition to enhancing the clinical problem-solving process. Incorporating role-play, team-based learning, and stimulated-patient encounters as teaching methods have been highlighted in the literature and proved to affect participants’ learning experience positively.31,64,66-69

Literature has identified students’ aspiration for instructors to adapt to students’ needs by implementing meaningful examples to invoke discussions related to practice and in doing so motivate students to do their best.70 The instructor of this course immersed students in the online course via video conferencing, providing related examples, discussions, workshops and motivational feedback. Students who participated in this study considered these methods useful and demonstrated enhanced confidence in handling clinical cases that might be encountered in practice. Hence these approaches of teaching were selected to be incorporated as components supporting the traditional didactic lectures component at baseline, in our course, with good outcomes for the students.

For the first time, this study explored the implementation and evaluation of customised educational intervention that aims to enhance students’ knowledge and skills in clinical problem-solving process in Jordan. The findings have demonstrated the remarkable positive impact of this educational intervention on decision-making skills of participating pharmacy students. The study highlighted in particular, a significant increase in participants’ confidence level in decision-making after joining the course. The students’ feedback, also, instruct us about the perceived need/utility, impact and suggestions to be considered in the future regarding the delivery of this course.

Furthermore, this study demonstrated that the delivered standard education in the clinical pharmacy and therapeutics tutorials was quite sufficient to provide the pharmacy students with the needed knowledge and skills to demonstrate the clinical problem-solving process. Such interventions were considered effective in helping students attain basic skills such as teamwork, peer assessment, communication and critical evaluation.

To serve the patients requirements, ample opportunity should be given to the students to develop their clinical knowledge foundation and improve their communication skills effectively. It is equally important to install in students’ positive attitudes and motivation to provide PC.37 This study highlighted the need for incorporating such courses in the undergraduate curricula in all universities in Jordan.

The findings of this study need to be interpreted with caution because of some limitations. First of all, the effectiveness on real practice was not assessed, and students’ skills in the clinical problem-solving process were evaluated only using few cases scenario that applies to the hospital/clinic settings. To resolve many of the obstacles recognized in this study, facing pharmacy students with real cases and providing them with a practical educational approach to deliver counselling to these real patients are required.71,72 Further investigation is still needed to find ways, with enhanced educational and training strategies, that overcome the identified barriers revealed in this study. These ways should be able to improve students’ confidence and competency in patient counselling once they start practicing pharmacy.

Also, our study was conducted only among one cohort of students from one university which may limit the generalizability of the results. However, no major differences can be found in the other schools of pharmacy that can limit the generalization of the findings of this study and much can be learned from this initial trial.

This study was conducted in unprecedented COVID-19 pandemic circumstances, which mandated lockdowns and travel restrictions, thereby obligating the construct of the educational intervention to be in an online format and adding some complexity to delivery and evaluation. Online teaching has often been improvised rapidly in such circumstances.73 Yet despite the challenges, the delivery of this course conformed with aspects identified as essential components for efficient information delivery modes, yielding positive student outcomes focusing on competency development.73

Final limitation of this study is that students might have responded in a socially undesirable manner in the post-intervention survey, which we addressed by thoroughly inspecting responses and discarding irregular or “bogus data”.

CONCLUSION

This study explored the implementation and evaluation of an educational medical informatics tutorial that aims to enhance students’ knowledge and skills in clinical problem-solving process in Jordan. The results of this interventional study indicate that this educational intervention achieved a significant positive impact of on decision-making skills of participating pharmacy students. The study highlighted in particular, a significant increase in participants’ confident level in decision-making after joining the course. Furthermore, such intervention was considered effective in helping students attain basic skills such as teamwork, peer assessment, communication and critical evaluation.

Effective medical care improvement and PC requires the collaboration of healthcare providers in ensuring the accurate use of medications through transitions in care. Among other healthcare professionals, pharmacists as medication experts have a vital role contributing in the success of process implementation. Pharmacy educators are liable for preparing pharmacy student to be ready to implement PC in their future practice.