The applicability of a validated team-based learning student assessment instrument to assess United Kingdom pharmacy students' attitude toward team-based learning.

PURPOSE: It aimed at testing the validity and reliability of a validated team-based learning student assessment instrument (TBL-SAI) to assess United Kingdom pharmacy students' attitude toward TBL.
METHODS: TBL-SAI, consisting of 33 items, was administered to undergraduate pharmacy students from two schools of pharmacy each at University of Wolverhampton and University of Bradford were conducted on the data, along with comparison between the two schools.
RESULTS: Students' response rate was 80.0% (138/173) in completion of the instrument. Overall, the instrument demonstrated validity and reliability when used with pharmacy students. Sub-analysis between schools of pharmacy did, however, show that four items from Wolverhampton data, had factor loadings of less than 0.40. No item in the Bradford data had factor loadings less than 0.40. Cronbach's alpha score was reliable at 0.897 for the total instrument: Wolverhampton, 0.793 and Bradford, 0.902. Students showed preference to TBL, with Bradford's scores being statistically higher (P<0.005).
CONCLUSION: This validated instrument has demonstrated reliability and validity when used with pharmacy students. Furthermore students at both schools preferred TBL compared to traditional teaching.

Introduction

Team-based learning (TBL) developed by Larry Michaelson in the late 1970’s for business education has been adopted in health professional education, in particular, medical education. The first reported use of TBL in pharmacy education was in 2008 [1]. It has since been adopted in a number of colleges and schools of pharmacy, predominantly in the United Sates [2] but more recently in the United Kingdom [3,4]. The majority of papers from pharmacy and medical education have shown improvement in marks/grades for that particular course of study and an increase in overall cohort progression, which is also accompanied with generally positive student perception [5,6]. To the authors’ knowledge there is no published validated instrument within medicine or pharmacy to TBL in assessing student opinion. Given the increased use of pharmacy, the ability to use a validated tool regarding the use of this teaching methodology is both timely and needed. The only example of a validated instrument to gather student perceptions of TBL comes from nursing [7]. The team-based learning student assessment instrument (TBL-SAI) consists of 33 items using 5-point Likert scale and contains 3 subscales. Subscale 1 measures accountability (items 1-8, a score of 25 or more favors TBL, in terms of students preparing for class and contributing to a team). Subscale 2 measures learning preference (items 9-24, score of 49 or more indicates preference in favor of TBL). Subscale 3 measures student satisfaction (items 25-33, a score of 28 or more indicates student satisfaction in favor of TBL). Therefore this study aimed at determining if the TBL-SAI can be used with pharmacy students validly and reliably.

Methods

Students from two schools of pharmacy, at Wolverhampton and Bradford in the United Kingdom, participated in the quantitative survey study where TBL was in 2011 and 2012, respectively. At both schools TBL was used to deliver material on diagnostic reasoning and managing signs and symptoms seen in a community pharmacy. This represented one sixth of each respective year’s teaching (equivalent to 20 credits worth of learning). For Wolverhampton, TBL took place in the third year whereas at Bradford this occurred in the fourth year of study.

The TBL-SAI available from: http://links.lww.com/NE/A75 was administered to Bradford students in 2013 and to Wolverhampton students in 2015. Permission was granted by Heidi Mennenga to use the instrument. Students from both schools completed the survey ‘in-class’ at the end of teaching and before any final examinations were undertaken. Data was collected anonymously. Survey data were entered into IBM SPSS ver. 20.0 (IBM Co., Armonk, NY, USA) and analyzed using descriptive statistics and the following statistical tests; Kaiser-Meyer-Olkin (KMO) principal axis factoring with varimax rotation and independent samples t-test. Sample size estimation and post hoc power analysis was determined using GPower software ver. 3.1.9.2 (GPower Software Inc., Kiel, Germany).

Ethical approval

Ethical approval was granted by the Behavioural Sciences Ethics Committee at the University of Wolverhampton and the Biomedical, Natural, Physical and Health Sciences Research Ethics Panel at the University of Bradford.

Results

The total response rate was 80.0% (138/173): Wolverhampton, 74.1% (63/85) and Bradford, 85.2% (75/88). The power was 0.83 using post hoc power analysis, with a sample size estimation of 84 (with effect size=0.8).

Data analysis

The data was subject to the KMO measure of sampling adequacy prior to factor analysis being performed. Values greater than 0.60 for the total instrument indicate that factor analysis can be performed [8]. The KMO score for the total instrument was 0.846 (Wolverhampton, 0.675; Bradford, 0.767) allowing factor analysis.

Validity and reliability testing

Principal axis factoring with varimax rotation was performed on the data (Appendix 1). Items of which factor loading values higher than 0.40 were shown to be valid within the instrument [9]. The results showed that all items had a factor loading of greater than 0.40. Sub-analysis of the data showed that for Wolverhampton, items 4, 21, 28, and 30 had factor loadings of less than 0.40 (Table 1), whereas no items in the Bradford data had factor loadings less than 0.4. Internal consistency tests were also conducted using Cronbach’s alpha to determine reliability of the total instrument. Cronbach’s alpha score was reliable at 0.897 for the total instrument: 0.793 for Wolverhampton and 0.902 for Bradford.

Table 1.

Factor loading of less than 0.40 for detected items for varimax rotation from questionnaire of team-based learning student assessment instrument provided to pharmacy students in University of Wolverhampton, United Kingdom

No.	Items	Factor
		1	2	3	4	5	6	7
4	My contribution to the team is not important.	-0.479		-0.341	0.103		-0.123	0.313
21	I can easily remember from lecture.			0.126	-0.582	0.148		0.236
28	I do not like to work in teams.	-0.722		0.122	0.263		0.206
30	Team-based learning activities are a waste of time.	-0.754		0.150	0.214	0.196		-1.01

Comparison of data sets

The total instrument score and the scores for each school of pharmacy along with the three sub-scales scores is shown in Table 2. Total scores and all three sub-scale scores showed preference to TBL. Sub-analysis showed that scores at Bradford were higher than those at Wolverhampton (P<0.005). The average percentage score for the TBL modules in the study was 64.0% from the students in Wolverhampton and 67.0% from students in Bradford. The average percentage score across the year was 61.6% from students in Wolverhampton and 67.0% from students in Bradford. Although no statistical analysis was performed on the attainment data, the average percentage module scores and year scores appear to be very similar.

Table 2.

Total TBL student assessment instrument score and sub-scale scores for schools of pharmacy at University of Wolverhampton and University of Bradford, United Kingdom

	Accountability (reference range: >25 favors TBL)			Preference to teaching style (reference range: > 49 favors TBL)			Students' satisfaction (reference range: > 28 favors TBL)			Total (reference range: >102 favors TBL)
	Range	Mean	SD	Range	Mean	SD	Range	Mean	SD	Range	Mean	SD
Total	8-40	30.7	5.4	30-76	54.2	8.8	13-45	33.0	7.1	63-158	117.9	17.9
Wolverhampton	13-35	27.7	4.3	30-62	50.2	6.2	17-37	28.8	4.7	80-131	106.7	11.4
Bradford	8-40	33.2	5.0	33-76	57.6	9.2	13-45	36.5	6.8	63-158	127.2	19.7

TBL, team-based learning; SD, standard deviation.

Discussion

The primary purpose of this study was to determine the applicability of the TBL-SAI when used with pharmacy students. Statistical analysis, through factor analysis, showed that all items had a factor loading of greater than 0.4; therefore, the TBL-SAI appeared valid and reliable when used with pharmacy students to assess their attitude towards TBL. Given the increasing use of TBL as a teaching method within pharmacy education, the ability to use a validated tool will give educators the opportunity to test student preference. Data sets from both schools showed preference for learning using TBL compared with traditional methods. The majority of students enjoyed and benefited from learning using this methodology. These findings mirror those seen in nursing students [7].

Preference score for TBL from Bradford was higher than those from Wolverhampton. This difference may be partly explained through differences in instructional delivery of TBL, given that the module content was very similar. Bradford adopted the ‘purist’ approach to TBL. They had dedicated collaborative learning rooms, incorporated summative peer evaluation as part of the process and summatively assessed the readiness assurance tests. In contrast Wolverhampton, had no dedicated TBL teaching facility, and did not have peer evaluation in place as the sessions were formative in nature.

A further difference in delivery was that multiple staffs with specific TBL training delivered material at Bradford compared to a single individual at Wolverhampton. Wolverhampton has subsequently developed a purist approach to TBL, with evaluations ongoing. In addition to delivery methods, the students were at different stages of their education. Bradford students were one year ahead and had much greater exposure to experiential learning on managing patient signs and symptoms than Wolverhampton students. Notwithstanding these differences, overall, the tool showed validity in both schools’ students lending some weight to the applicability of its use. We acknowledge that comparing the two groups does have limitations to the data but both schools were chosen as they were early adopters of TBL methodology in United Kingdom pharmacy education and gave the opportunity to assess the usefulness of the TBL-SAI allowing other educators to potentially benefit from our experiences. Mitigating against the use of two differing groups was that they appear to have similar academic profiles; therefore, the differences in TBL preferences is more likely due to the TBL delivery rather than the students’ differences.

Two groups of pharmacy students were included in the study, with both being relatively small in terms of sample size. The students in Wolverhampton had a slightly lower response rate than those in Bradford, which may have had an impact. Finally, the study only demonstrates preference of TBL and not of academic outcomes of TBL compared to traditional teaching. Therefore, it is recommended that further testing is required in pharmacy schools that are using TBL to further add to evidence of higher academic outcome.

In conclusion, the TBL-SAI demonstrated reliability and validity in pharmacy students. Students in both pharmacy schools preferred TBL compared to traditional lectures, although Bradford students have a much stronger preference for TBL. The tool appears valid for use in pharmacy students but more widespread use of the tool is required to see if the results of this study are reproducible to determine adoption of the tool for pharmacy educators of TBL.