Metacognition and Peer Learning Strategies as Predictors in Problem-Solving Performance in Microbiology.

The objective of this study was to determine the relationship between learning strategies (LS) and problem solving (PS) in microbiology. Microbiology problems utilized for the study were from educational software known as "Interactive Multimedia Exercises" (IMMEX). Problem-solving performances measured included: the ability to solve, scores obtained and elapsed time. It was hypothesized that there would be a good correlation between students' LS and PS. Since many factors besides learning strategies predict performance, alpha was set at 0.10. Participants (N = 65) solved two sets of microbiology problems "Microquest" (Mq), which focuses on microbial cellular processes and mode of action of antibiotics, and "Creeping crud" (Cc), which focuses on the cause, origin, and transmission of diseases. Participants also responded to the adapted Motivated Strategy Learning Questionnaire (MSLQ) using a five-point Likert scale. Scores for LS were determined by averaging the item responses of participants. Regression analysis was used to determine significance, with Grade Point Average (GPA) as a control. Of the 65 participants 48 (73.8%) successfully solved Mq while 52 (80%) solved Cc. Metacognitive self-regulated strategy was significantly (p < 0.10) related to ability to solve Cc. Peer learning strategy showed a significant (p < 0.10) relationship with Cc scores. Time spent solving Cc was significantly more than time spent on Mq (p < 0.001). These findings emphasize the fact that metacognition and peer learning are positive predictors for problem solving and could potentially improve learning outcomes in microbiology. The implications for curriculum development are discussed. ©2020 Author(s). Published by the American Society for Microbiology.

INTRODUCTION

Understanding biological concepts requires problem solving, and it is therefore imperative to identify strategies used to solve problems successfully (1, 2). Problem solving means answering a question that cannot easily be retrieved from memory but must be constructed from information available. It can be characterized as a cognitive process that is goal-directed and requires effort and concentration (3). Metacognition in general means the self-monitoring of, and conscious use of learning strategies. It is not an automatic process but rather a result of long-term development of the cognitive process (3, 4). Metacognition has been shown to correlate well with the ability to solve problems (5, 6). Such findings are relevant to the focus of current educational emphasis because learning strategies can be learned and steps for teaching such different strategies can be developed (7, 8).

In education, a major emphasis is placed on problem solving. Successful problem solving can be accomplished using components of self-regulated learning that promote students’ metacognitive abilities (8). Bandura’s Social Cognitive Theory (9) provides the explanation that a learner’s decision-making process involves the ability to self-regulate and control behavioral responses. The theory views human functioning as reciprocal interactions between behaviors, environmental variables, and cognition. This theory explains that learners are not controlled by external views but rather possess self-directed capabilities to influence their own behavioral responses (10). In the standard information-processing theory of mental architecture, it is explained that there is an encoding process which involves a pattern that connects information in the working memory to previously acquired information in long-term memory (11). Through inquiry, case studies, scenarios, and availability of adequate technology and learning tools, students can use different strategies to solve real-life problems.

Unlike previous reforms and instructional theories that assumed that students play primarily a reactive role in the learning process, the goal of the current education system is to shift to the learner the primary responsibility of pursuing his/her own education (1, 2). Such abilities can be emphasized by providing learning opportunities for students to use critical thinking and other learning strategy skills. Curriculum development that involves metacognitive abilities during student engagement has the potential to enhance learning (12). Students have lived experiences and other notions and past knowledge on which they can build. Technology-based learning environments provide opportunities to perform studies and collect data so that suitable learning strategies such as metacognitive attributes can be measured (13).

Interactive multimedia exercises (IMMEX) (IMMEX Educational software www.immex.com ) is the software program that was used for this study. It is a Windows-based software system that contains many problem-based scenarios with a starting point, a goal, and numerous options to achieve the goal. When students get to a section they do not understand, they can log on to the library menu within the program for additional information (14). The IMMEX program puts the learner in the role of a scientist investigating a problem.

There are few quantitative studies that have used the learning strategy of learners to predict performance in microbiology using educational software at the college level. This might be due to the complexities of carrying out such studies. The theoretical framework for the current study was based on the constructivist social cognitive model, which assumes that learning strategies can be learned depending on the determination of the student (15). Therefore, the purpose of this study was to determine the relationship between learning strategies (LS) and problem solving (PS). The research question was, “To what extent do each of the nine different learning strategies (Table 1) relate to performance in problem solving microbiology problems in the IMMEX problem sets?” It was hypothesized that there would be a good correlation between students’ LS and their PS performance. Due to the number of behavioral factors that can affect performance outcomes, alpha was set at 0.10.

TABLE 1

Reliability of current study after modification compared with original Pintrich et al. reliability.

Variable	Current study	Pintrich et al.
Cognitive strategies
1. Rehearsal	0.69	0.69
2. Elaboration1	0.62	0.76
3. Organization	0.70	0.64
4. Critical thinking	0.77	0.80
Metacognitive strategy
5. Metacognitive self-regulation	0.83	0.79
Resource management strategies
6. Time and study management	0.83	0.76
7. Effort regulation	0.62	0.69
8. Peer learning	0.62	0.76
9. Help seeking	0.70	0.52

The rationale for conducting this study was to understand and re-emphasize the importance of incorporating good learning strategies to ensure success in the learning process. The study was conducted based on the social cognitive framework used to create the Motivated Strategy Learning Questionnaire (MSLQ). It assumes that learning strategies are under the control of the student and can be learned. Participants (N = 65) solved two sets of microbiology problems: Microquest (Mq), which focuses on cellular microbial processes and the mode of action of antibiotics, and Creeping Crud (Cc), which focuses on the cause, origin, and transmission of microbial diseases. Participants also responded to the adapted MSLQ, using a five-point Likert scale. Scores for LS were determined by averaging the item responses of participants. Regression analysis was used to determine the significant relationship between learning strategy and problem-solving performance using GPA as a control.

METHODS

Participants

Participants were undergraduate microbiology students (N = 65) from a mid-sized university in the Western United States who were required to take General Microbiology as part of their major in Biology (n = 38), Allied Health (n = 23) or one of two other science majors. They were taught by a qualified, experienced microbiology educator with 30 years’ experience. The IMMEX exercises were included in the General Microbiology laboratory spring schedule. Internal Review Board (IRB) approval was obtained from the institution where the study was conducted. At the beginning of the study, students were asked to complete a consent form that contains information about the researcher (Appendix 1).

Attached to the consent form was a brief description of the different phases of the study (Appendix 2). These were also verbally explained to the students. Any student who did not consent to the study would still have used the IMMEX software as part of the lab experience for the week but no study data would have been collected for research purposes. Phase 1 was the introductory phase. Phase 2 was the beginning of the main study. Phase 3 was the verbalization study (1) and is not included in this current study. Phases 1 and 2 were included in the spring laboratory schedule. In phase 1, in March, participants were introduced to the program by solving 2 practice problem sets: “Who messed with Roger Rabbit?” and “Puffy Paramecium.” The researcher circulated through the lab answering questions as students familiarized themselves with the program and the concept of answering questions about a real-world scenario. For the main study (phase 2), two new problem sets were included: during the week of April 9, participants solved Mq problems, which focus on microbial cellular processes and the mode of action of antibiotics. During the week of April 16, they solved Cc problems, which focus on the cause, origin and transmission of microbial diseases. All participants then answered the adapted MSLQ (1) during lab time after completing their assigned problem sets.

Problem solving using the IMMEX educational program

The microbiology problem sets, just like most IMMEX problem sets, have three major components: a prologue (Appendices 7 and 8), a problem space, and an epilogue. The prologue comes in the form of a narrative that provides students with a scenario. The problem space shows all menu item selections embedded in the problem. Menu items include: 1) interpretive information such as laboratory tests results, statistical data, topographic maps, graphs, and demographic data, 2) a library that provides students with background information to fortify students’ knowledge gaps, 3) help information such as hints and/or expert opinions, 4) a score showing starting, current and ending points. The epilogue provides a list of possible answers or solutions to the problem. The multiplicity of pathways in the IMMEX program helps to actually simulate authentic real-life problem-solving experiences. Each participant attempted three problems from each of the two problem sets (Mq and Cc). While attempting each problem (Fig. 2), every selection of an item from the sub-menus causes points to be deducted from a starting score of 1,000. Therefore a quick solution with minimal selection will yield a high final score. Likewise, a poor score results from many selections where the user has to search through many sub-menus to arrive at a solution to the problem. A graphical print that documents how students performed is available to the researcher.

FIGURE 2

Example of the experience of a Creeping crud problem solver. Available optional resources include Library, Experts, and Maps.

Learning strategy instrument

To determine the LS, a modification of the published instrument (MSLQ) (Appendix 3) was administered to participants. Five of the LS scales showed internal consistency estimates greater than 0.75 (Table 1). Items in the questionnaire measure different LS subscales. Twelve items make up the metacognitive self-regulation scale while three items make up the peer learning scale (Appendix 4). In the current study, reliability ranged from 0.63 (elaboration) to 0.83 (metacognitive self-regulation) among the cognitive and metacognitive strategies and from 0.70 (help-seeking) to 0.83 (time and study environment) among the resource management strategies (Table 1). The MSLQ authors (4, 15) report one confirmatory factor analysis (CFA) as evidence of the MSLQ’s validity (Appendix 4). The authors ran CFA models for learning strategy sub-scales and then for all the sub-scales combined. This was done to determine whether the structure of the measure is consistent with expectations. In the current study, some modifications of the initial instrument were made to improve internal consistency. For example, item 57 was removed for metacognitive analysis (Appendix 5). The reliability of the instrument was determined using Kuder Richardson’s method. To increase the reliability value, certain items were dropped (Appendix 5). Data that were analyzed in this study included average scores from the responses of participants to the modified MSLQ questionnaire (LS score).

Performance

The print sheet generated from the IMMEX software indicates participants who solved the problem and those who were unable to. Those who successfully completed at least one out of the last two questions assigned were categorized as “solved” while those who did not solve any were categorized “not solved.” The average scores of the last two problems (out of the three solved) were computed and used for analysis. The rationale for choosing the last two attempts was to be able to conduct repeated measures analysis. A within-subject repeated measures ANOVA analysis was performed, and the means for the three problems were compared. Paired t-tests were done to determine differences in scores. The last two problems were used for analysis because between-group analysis revealed a significant (p < 0.05) difference between the first attempt and the other two attempts for both the Mq and Cc. (16). Analysis using Wilks’s lambda (a test statistic used in multivariate ANOVA to test differences between means of groups) was used to determine the difference in time spent on Mq and Cc.

Learning strategies

The social cognitive theory on which the MSLQ is based assumes that responses by students would vary depending on the course. In other words, the same student can answer differently with reference to another course (15). Answers to questions that measure each learning strategy (LS) ranged from 1 (for strongly agree) to 5 (for strongly disagree). Appendix 3 shows the entire questionnaire including the demographics section. Appendix 5 shows the individual group of questions that measure each LS. To compute the number that represents each strategy, all participant responses were summed (using the Likert responses) and an average was computed to represent the number used for multiple regression analysis.

Quantitative data analysis

There were 10 independent variables of interest: 9 learning strategies (Fig. 1, Table 1) and the GPA. The latter served as a control because it was not of primary interest in this study. There was only one dependent variable: performance. Two sub-variables were used to measure performance: one categorical variable (solved or not solved) and two continuous variables (scores and time spent).

FIGURE 1

Schematic representation of study design.

Multiple regressions analysis

Studying a construct or a variable requires identifying the sources of variation. A variable is defined as any attribute upon which objects or individuals vary. Therefore when an instrument is applied to measure a variable among several individuals, different scores will be relatively obtained for each individual, for example, the variance of college GPA (as a measure of achievement) or the variability among individuals on a scale designed to measure learning strategy, locus of control, or strength (17). Science attempts to explain this variability. To explain the variability of a phenomenon of interest (dependent variable) scientists attempt to study its relation or covariation with other variables (independent variables). In this study, the researcher attempted to explain the role that certain LS play in explaining PS performance. Multiple regression analysis is suitable for analyzing the collective and separate effects of two or more independent variables on a dependent variable of interest.

Multiple regression equation

The result of multiple regression and correlation (MRC) analysis provides the weighted linear combination of predictors (independent variable) that, based on the least square criteria, shows the best prediction or explanation of the criterion. Both raw and standard scores are used in the equations.

Raw score multiple regression equation:

Standard score multiple regression equation:

Where Y’ and zy’ denote the predicted criterion (dependent variable), X and Z represent the predictors (independent variables), B represents the raw partial regression coefficient, β represents the standardized partial regression coefficient, and a is the intercept.

Multiple regression in explanatory research

Multiple regression analysis was used to determine how much variance in the dependent variable could be explained by a set of independent variables collectively. Since the dependent variable (performance) was measured in different ways, the continuous independent variables (scores and time) were analyzed using hierarchical multiple regression. The categorical independent variable (solved or not solved) was analyzed using hierarchical logistic regression.

Hierarchical regression analysis

According to this approach, the proportion of variance accounted for by all independent variables (i.e., R2) is partitioned incrementally. In this analysis, the order of entry of variables into the analysis is crucial. Generally, this approach is used to study one of the following: 1) the effect of an independent variable on the dependent variable after having controlled for another variable, and 2) the relative effects of a set of independent variables on the dependent variable (17).

In the current study, the controlled variable that was entered into the regression analysis was the GPA of each student. It was important to know the effect of each of the nine independent variables in this study after controlling for the effect of GPA. Each component of performance (solved vs. not solved, score, time) was used to determine whether any relationship existed with the independent variables after controlling for GPA. To interpret the relative contribution of each independent variable, partial coefficient values from the coefficient table were used as well as significance level. Partial coefficients tell us the unique effect of a predictor after controlling for the effect of all other predictors (18). In general, two forms of the coefficient are available. The first is called the raw score partial regression coefficient (symbolically denoted as B, see Eq. 1) and standard score partial coefficient (symbolically denoted as B, see Eq. 2). Average performance for the last two of the three problems solved for each problem set was computed for each sub-variable. Likewise, to obtain average time, the elapsed time on the last two of the three problems solved was used. A hierarchical multiple regression analysis was performed to determine the relationship of each sub-variable with each of the independent variables. The first independent variable that was entered into the analysis step was GPA.

Hierarchical logistic regression analysis

To explain the role of learning strategies and performance using solved or not solved, hierarchical logistic regression was used. This approach was chosen because the dependent variable is categorical. The dependent variable has only two values: the ability to solve or the inability to solve (as defined in this study). Like linear regression, the model can relate one or more predictor variables to a dependent variable. The model yields regression coefficients, predicted values, and residuals. In logistic regression, the predicted values (dependent variable) and the predictor (independent variables) are assumed to be nonlinear (17). Students who solved are arbitrarily coded as 1 and those who did not are coded as 0. The curve obtained never goes below 0 or beyond 1, and the predicted values can, therefore, be interpreted as probabilities.

Another important concept in logistic regression is the odds ratio, which estimates the change in the odds of membership in the target group for a one-unit increase in the predictor (18). In interpreting information in logistic regression, the raw coefficient of the predictor variable is the change in the natural log of the odds ratio (Exp(B)). A positive predictor coefficient means that the predicted odds increase as the predictor values increase. If the raw coefficient is negative, it means the predicted odds decrease as the predictor increases (inverse relationship) (18). A coefficient of zero indicates the predicted odds are the same for any predictor values. This means the odds ratio is 1.

RESULTS

Table 1 shows the reliability results of the MSLQ, the instrument used to determine the learning strategy. Reliability ranged from 0.62 (elaboration) to 0.83 (metacognitive self-regulation) among the cognitive and metacognitive strategies and from 0.70 (help-seeking) to 0.83 (time and study environment) among the resource management strategies. Table 2 provides the characteristics of the study participants. A total of 65 students participated in the study. The majority were juniors (n = 26 [43%]). Similarly, more females (n = 46 [78%]) than males (n = 13 [22%]) participated. The age ranges were 18–20 (46%), 21–24 (28%), and 25–30 (9.2%). The number of students who could solve Mq was 48 (73.8%), while 52 (80%) could solve Cc. Table 3 shows the average LS score from MSLQ computed responses and the performance criteria (solved or not solved, scores, and time spent) data computed from IMMEX problem sets. The average score for Mq was 619.32 while that of Cc was 630.9 after deductions from 1,000 starting points. This analysis was based on those who solved (N = 48 for Mq and N = 52 for Cc). In this study, the research question was “To what extent do each of the nine different types of learning strategies (LS) relate to performance in problem solving microbiology problems in the IMMEX problem sets?” It was hypothesized that there would be a positive correlation between students’ LS and performance in PS. Hierarchical logistic analysis was used for the categorical PS, solved or not solved. The other performance criteria (scores, time spent) were analyzed using hierarchical multiple regression. Tables 4 and 5 show the regression analysis of the relationship between LS and PS. Table 4 shows that metacognitive self-regulation has a significant positive correlation (p < 0.1) with the ability to solve Cc. The Exp(B) value is more than one (7.720) and B has a positive value (2.044). The table shows a significant value of 0.069 (Appendix 6, SPSS stats print). The odds are 7.720 times greater for someone with metacognitive skills to solve Cc than someone without. Also, in Table 4, there is a significant inverse relationship (p < 0.1) between help-seeking and ability to solve Mq. Note that Exp(B) is less than 1 (0.340) and B has a negative value (−1.080) and a significance of 0.055. The more help sought, the worse the performance. The GPA, which was used as a control, was statistically significant (p = 0.044) in the ability to solve Mq (data not shown), but GPA did not predict a good relationship with Cc. Table 5 presents values for the hierarchical multiple regression analysis. It shows a positive significant correlation (p < 0.1) between peer learning strategy and the ability to solve Cc. No statistical correlation was observed between the other strategies and PS. Of all the learning strategies measured by the MSLQ, metacognitive strategy and peer learning strategy showed significance in predicting success in solving Cc. Average time spent on PS was significantly higher for Cc (8.64 minutes) than Mq (4.41 minutes).

TABLE 2

General characteristics of study participants.

Characteristic	N (%)
Gender
Male	13 (22)
Female	46 (78)
Year in college
Freshman	1 (1.6)
Sophomore	21 (35)
Junior	26 (43)
Senior	12 (20)
Age range
19–20	30 (46)
21–24	18 (28)
25–30	6 (9)
>30	11 (17)

TABLE 3

Average learning strategy (LS) score from Questionnaire Likert scale.

	Microquest (N = 48)		Creeping Crud (N = 52)
Learning Strategy (LS)	Mean	SD	Mean	SD
Rehearsal	3.42	0.78	3.43	0.74
Elaboration	3.61	0.47	3.62	0.55
Organization	3.14	0.74	3.18	0.84
Critical thinking	2.87	0.70	2.94	0.71
Metacognitive self-regulation	3.2	0.58	3.26	0.63
Time and study environment	3.3	0.74	3.29	0.74
Effort regulation	3.0	0.48	3.03	0.48
Peer learning	2.69	0.86	2.69	0.85
Help seeking	3.18	0.95	3.22	1.0

TABLE 4

Hierarchical logistic regression analysis of learning strategies.

	Ability to Solve Mq N = 48				Ability to Solve Cc N = 52
Variable	B	df	Sig	Exp(B)	B	df	Sig	Exp(B)
Rehearsal	−0.145	1	0.814	0.865	169	1	0.801	1.184
Elaboration	−0.467	1	0.622	0.627	−1.197	1	0.279	0.302
Organization	0.559	1	0.400	1.749	0.275	1	0.670	1.316
Critical thinking	0.347	1	0.595	0.707	0.394	1	0.520	1.484
Metacognitive self-regulation	0.545	1	0.621	1.724	2.044	1	0.069a	7.720
Time and study environment	0.008	1	0.991	0.992	−2.10	1	0.129	0.298
Effort regulation	−0.497	1	0.586	0.609	k.148	1	0.880	1.155
Peer learning	0.111	1	0.831	1.118	0.178	1	0.715	0.837
Help seeking	−1.080	1	k0.055b	0.340	−0.433	1	0.336	0.649

p < 0.10 (alpha set at 0.1) 7.720 times greater chance of being able to solve Cc with metacognition as learning strategy.

p < 0.1 (inverse relationship) (alpha set at 0.1), Note: Exp(B) shows <1 (0.340) and B is negative (−1.080). The more help sought, the worse the performance

TABLE 5

Hierarchical multiple regression analysis of learning strategy vs. scores for Cc (N = 52).

Model	B	sig	Partial	Part
Rehearsal	36.365	0.51	0.102	0.095
Elaboration	117.859	0.185	0.206	0.195
Organization	−33.832	0.462	−0.115	−0.107
Critical thinking	−13.430	0.787	−0.043	−0.039
Metacognitive self-regulation	−55.40	0.468	−0.114	−0.106
Time and study environment	−36.904	0.495	−0.107	−0.099
Effort regulation	28.585	0.671	−0.067	−0.062
Peer learning	75.080	0.084a	0.266	0.255
Help seeking	−33.673	0.362	−0.142	−0.133

p < 0.1 (alpha set at 0.1)

DISCUSSION

The researcher investigated the relationship between LS and PS among students taking a Microbiology course. Microbiology problems used for the study were from educational software known as Inter Multimedia Exercises (IMMEX). It was hypothesized that there would be a good correlation between students’ LS and PS. Since there are many ways that performance can be measured, the parameters used in this study were the ability to solve the problem, the scores obtained, and the time elapsed. At the outset, the reliability of the learning strategy was too low and some items were therefore removed to increase reliability (Appendix 5). The study shows that metacognitive strategy (p < 0.1) is important in the ability to solve Cc (Table 3), the data indicating that there was a 7.72 greater odds of solving Cc for someone who had metacognitive abilities than someone who does not. This finding was consistent with Martinez (5), who explained that a common feature of problem solving was metacognition. Zhao (6) showed that students who were trained with metacognitive skills outperformed those who were not. An unusual interesting finding was that help-seeking (Table 4) had an inverse relationship on performance in the ability to solve Mq. In other words, individuals who used help-seeking (a resource management strategy that refers to seeking help from peers, instructors, or other mentors) as a major learning strategy did more poorly in solving the Mq problems. It must be stressed that the reliability for help-seeking in this study was originally only 0.37 but after deleting questions 40 and 58, it increased to 0.70. Questions 40 and 58 were removed because the validity using the confirmatory factor analysis data was quite low for these questions (Appendix 5). This means the questions were not measuring the help-seeking strategy construct. A negligible correlation between help-seeking and course grade was previously noted by some investigators (19). Karabenick and Knapp (20) also observed significant negative help-seeking correlations with college students’ course grades.

Using hierarchical multiple regression to continue to investigate the research question, it was shown that peer learning (a resource management strategy that refers to collaboration with one’s peers) showed a positive significant (p = 0.08) relationship with scores obtained from solving Cc (Table 5). Hoffman (21) introduced collaborative learning among peers to her microbiology class, and the program was successful, judging from performance and students’ evaluation of the method. Also, Trottier (22) found success in certain sections of pharmacology with cooperative learning.

Analysis using Wilks’s lambda showed a statistical difference between time spent to complete Mq compared with Cc. The time it took to complete Cc was statistically significantly higher p < 0.001 compared with the time to solve Mq.

CONCLUSION

Overall, there was no significant difference in the difficulty between the two problem sets. However, time spent was significantly higher for Cc than for Mq, suggesting that time could be dependent on the type of microbial problem posed and/or how clear the instructions for solving the problems are. The study shows that metacognitive strategies were significant (p < 0.10) in the ability to solve Cc, suggesting that it may be useful to incorporate them in planning educational curricula. This would allow students to plan, think through, and develop critical thinking skills. Incorporating activities that make students think and solve problems such as context simulation enhances students’ problem-solving skills (7).

Peer learning strategy showed a significant (p < 0.10) relationship with scores obtained in solving Cc. This suggests that collaborative learning can lead to better outcomes. Student engagement techniques assist in the learning process. There is a growing concern that our current education system may not be preparing students for the workforce of today (16, 23). Findings from this study show that peer learning is a likely predictor of success. Therefore, educators are encouraged to consider incorporating collaborative problem-solving (CPS) modules in their curricula because, in the current workforce, employees collaborate during face-to-face interaction as well as online using technology that crosses boundaries (16, 23).

Strengths and limitations

The findings of this research are indeed very timely. Microbiology is a complex subject that encompasses different biological disciplines and thus requires different combinations of learning strategies and problem-solving skills. Careful planning of the syllabus and activities based on the discipline or context is required because one size does not fit all.

Perhaps more strategies would have shown positive relationships if the sample size was greater. The analysis was based on only those who solved the problems, further reducing the number for analysis (Mq = 48 and Cc = 52). While the sample size for this study was adequate (24), a larger sample may have provided better outcomes. Also, the multiplicity of pathways available through IMMEX (IMMEX Educational software www.immex.com) may have made it difficult to conduct a straightforward analysis.

Another limitation is the assumption that participants were truthful in their responses to the MSLQ instrument. It was also not possible to determine prior knowledge. All participants were enrolled in a general microbiology course. The topics in the IMMEX program were already taught to the students before the IMMEX exercise.

Click here for additional data file.