The role of goal orientations and learning approaches in explaining digital concept mapping utilization in problem-based learning.

Concept mapping has received increasing attention and application in higher education as an effective instructional strategy. However, little is known about how higher-education students' different motivations for learning might be related to the way they use digital concept mapping for effective learning. This study sought to design and assess an intervention program that employs digital concept mapping in problem-based learning and to evaluate the effectiveness of using this tool among students with different achievement-goal profiles on learning and deep versus surface approaches to learning. Data were collected from 129 undergraduate students from three higher-education institutions located in Israel and Austria and analyzed by using Partial Least Squares - Structural Equation Modeling. The findings indicated that digital concept mapping could benefit higher-education students, specifically at the cognitive level, in order to specify and identify the interrelationships among arguments and to learn about the topic. Another finding showed that deep learners and mastery-approach learners perceived concept mapping as an effective tool mainly for self-regulating their learning during the intervention. It is suggested to find ways to scaffold surface learners' involvement in activities that enable them to solve complex problems by underlining the benefits of technology-enabled platforms for their learning and thus have them acknowledge concept mapping as a practice that fosters meaningful learning.

Introduction

The COVID-19 global pandemic has affected institutions of higher education, obliging the adoption of technology-delivered instruction. The shift to remote learning has underscored the growing need to reconstruct learning outcomes and adapt them to the current conceptual era. This era necessitates higher-level thinking skills such as the ability to identify patterns or connections between situations, creatively solve ill-structured problems, and identify underlying issues in complex situations (McWilliam, 2017). To address these skills, this study sought to design and assess an intervention program that employs digital concept mapping (CM) in problem-based learning (PBL) with a designated decision-making process and to evaluate the effectiveness of using CM in this process.

A central aim was to understand how students with different motivations for learning might perceive the effectiveness of using digital CM in a decision-making process required of them during a PBL activity. Notably, individual differences can determine the level of willingness or reluctance to use a new teaching/learning technique (Alt, 2021; Schaal, 2010). Therefore, the question of how technology-enabled CM’s utilization might vary with different ability learners (Schaal, 2010; Sun & Chen, 2016) lies at the core of the present study. Addressing this question might expand upon empirical results of previous studies which mainly measured the impact of educational activities with CM on variables such as students’ epistemic beliefs (Alt & Kapshuk, 2021) and self-regulation for learning (Alt & Naamati-Schneider, 2021). The novelty of this work lies in initially assessing students’ goal settings and approaches to learning as motivational precursors in the overall evaluation of digital CM in PBL pedagogical technique. The exploration of these motivations is especially important to address in academic settings that employ technology so that teachers could help different ability learners to exploit opportunities created by technological innovation to facilitate effective learning processes.

This study hinges on two learning motivation theories: achievement goals (Elliot & Murayama, 2008; Pintrich, 2000) and deep versus surface approaches to learning (Biggs et al., 2001), and seeks to examine how these precursors might be connected to students’ perceptions of the effectiveness of using CM in the decision making required in an ill-structured problem-based activity. Illuminating these associations could increase our understanding of how to use technology-enabled CM effectively for students with different motivations for learning in higher education learning environments.

Literature Review

Concept Maps for Constructivist Problem-Based Learning

The idea of CM (Novak & Gowin, 1984) is based on the constructivist approach to learning, which highlights the active role played by learners in constructing and developing knowledge and extracting meanings from their prior knowledge, experiences, and beliefs. Concept mapping is also often linked to Ausubel’s (2000) assimilation theory of meaningful learning (Roessger et al., 2018). Ausubel emphasized the centrality of relating concepts within a cognitive structure in learning and thinking processes. The cognitive structure pertains to the learner’s organization and clarity of knowledge in a particular topic. In a process termed assimilation, he pointed to learners’ proclivity to relate novel ideas to existing cognitive structures so as to reduce the meaning of the novel idea to that of the established one. Students construct knowledge meaningfully when they organize knowledge hierarchically from general to specific and use analysis and synthesis techniques to sort and relate concepts.

Drawing on these theories, Novak and Gowin (1984) introduced the idea of CM which is a visual representation of knowledge. The process of CM enables one to organize and structure information and the relationships between conceptswithin a particular domain. This may be done in a graphical manner (i.e., using images, photos, colors, etc.) to highlight various concepts and their linkages; or, alternatively, by identifying key concepts or ideas by names or titles, enclosing them in visual boxes, and labeling links between them (Jennings, 2012). Concept maps may aid the instructor in assessing what students understand and how they link the material to the overall course goals. Concept maps are easily taught and can be incorporated in introductory units, midterm reviews and assessments, peer assessment, self-assessment, and/or end-of-course reviews and assessments (Croasdell et al., 2003).

In their review, Machado and Carvalho (2020) have recently shown the benefits and challenges of using CM in higher education settings. Their work showed that CM has positive effects on students’ critical thinking and their ability to integrate theory and practice. CM enhances collaborative learning and enables technology inclusion and meaningful learning. Among these benefits, critical thinking and problem-solving are considered central abilities required for learners in the twenty-first century (Partnership for twenty-first Century Skills, 2014). Framing, analyzing, and synthesizing information to solve problems are considered important skills in this context. In higher-education settings, recent studies (Aliyari et al., 2019; Chan, 2017) have concluded that CM could provide learners with better and deeper learning instead of the superficial learning exemplified by traditional learning. In stark contrast to conventional lecture-based teaching methods, CM is a student-centered method. According to Motjolopane (2021), this method is deemed important particularly in the current Coronavirus global health emergency. The swift shift from frontal teaching into distance learning, forced on educational institutions, required placing high trust in technology-enabled learning and build on student engagement, without diminishing the quality of learning. In this context, CM has been repeatedly shown to have a positive impact on the quality of student learning (Schroeder et al., 2018). It does so by improving students’ critical thinking (Yue et al., 2017), decision-making skills (Huang et al., 2017), abstract reasoning (Cañas et al., 2017), meaningful learning levels (Bressington et al., 2018), and creativity (Xu & Pang, 2020) than other conventional teaching methods.

A recent study (Aliyari et al., 2019) examined the effects of CM-based and lecture-based teaching methods on the level of nursing students’ learning in Basic Life Support (BLS), by employing a quasi-experimental study including 57 students from a nursing school in Tehran, and found that the CM-based teaching method was significantly more effective than the lecture method. Similarly, Chan (2017) explored how students develop concept maps and what these concept maps consist of, as well as their views on the use of concept maps as a learning activity in a PBL class. Specifically, she investigated the issue of whether and how innovative concept maps supplemented with art elements improve students’ creative thinking abilities. The study was conducted at the school of nursing in a Hong Kong university. The author demonstrated how concept maps can be implemented in a PBL class to elevate the students’ creativity, active participation, and motivation to learn, and thus should be considered an alternative way to facilitate student learning.

Hung and Lin (2015) examined the patterns of concepts and differences in the knowledge structures of 52 occupational therapy undergraduates taught with, and without, a PBL approach. Three categories of concept maps were identified as isolated, departmental, and integrated. More isolated maps were designed by students in the control group, whereas more integrated maps were constructed by students in the PBL group. The authors concluded that PBL has a strong effect on the acquisition and integration of knowledge.

In the field of teacher education, Leumann (2017) demonstrated how concept maps can be used as a methodological and targeted instrument for the exploration of the deep structure of teachers’ conceptions with respect to aspects of specific phenomena and processes related to teaching, and therefore have great potential for research and teacher education. Campbell (2016) underscored the benefits derived from adding this learning tool to a teacher’s instructional toolbox and invited them to adopt digital CM into the social studies classroom “as a means… to provide opportunities for students to make relevant and important connections within the body of knowledge defined as social studies” (p. 74).

Aristika and Juandi (2021) underscored the importance of tailoring an appropriate technological tool to an appropriate learning model in a way that supports knowledge construction and enables opportunities for students to get actively and creatively engaged in the learning processes. Based on this notion, drawing on the constructivist approach, this study assesses the use of concept mapping in problem-based learning,

Concept Mapping and Individual Differences

The body of literature examining the relationships between CM and learning motivations of students has been somewhat undermined by inconsistent findings. While several researchers (e.g., Guthrie et al., 2007; Hsu, 2019) have suggested that CM can increase learner motivation, others asserted that the effects of this technique might vary among different ability learners (Schaal, 2010; Sun & Chen, 2016).

In line with the first premise, several researchers maintained that the effective use of CM can facilitate a learning environment that motivates students for learning. For example, Guthrie et al. (2007), showed how a Concept-Oriented Reading Instruction (CORI) can be designed to increase students’ reading comprehension and intrinsic motivation for reading. Hsu (2019) found that a preparatory CM task for high-school students improved four elements of learner motivation – attention, relevance, confidence, and satisfaction- and encouraged instructors to use CM to support learners in English-speaking activities with native and non-native speakers. Similarly, Martin et al. (2015) investigated published reports of empirical research studies that tested the effects of concept maps on the achievements of a wide variety of learners. The literature review, which included 55 refereed journal articles, provided useful information for adult literacy practitioners interested in utilizing CM as an approach to teaching and learning in their classes. Among several outcomes, CM demonstrated a significant positive effect on learners’ motivation.

Clearly, the study of these CM motivational outcomes has borne much fruit in the field of education. However, empirical studies that explored the potential role of such constructs as precursors, rather than outcomes, in conjunction with CM are scarce. Nonetheless, several researchers maintained that the effects of educational technique might vary with different ability learners and can be more effective for learners with higher levels of motivational goals compared to those who place little value on the learning tasks. For example, Schaal (2010) argued that self-efficacy might moderate the application of a meta-cognitive technique like CM; hence, students who believe they are capable of using concept maps are more likely to participate in this activity. Similarly, Sun and Chen (2016) showed how the use of concept maps significantly increased elementary-school students’ self-efficacy for learning when their initial self-efficacy was already high. They suggested that this type of teaching/learning strategy is more suitable for improving these students’ self-efficacy rather than students with low self-efficacy beliefs. For learners who believe in their ability to acquire knowledge, they suggested that teachers adopt dynamic concept maps in their educational materials and questions to promote learners’ thinking and enhance their learning achievements.

Notably, individual differences can determine the level of engagement, satisfaction, and willingness or reluctance to use a new teaching/learning technique and are deemed important criteria in the personal overall evaluation of a meta-cognitive technique like CM (Schaal, 2010). Therefore, understanding deeply the predictive power of individual differences on the effective use of concept maps during the learning process in higher-education settings is of the utmost importance. The present study investigated learning motivation from two perspectives: the achievement goal theory (Pintrich, 2000) and deep versus surface learning (Biggs et al., 2001). Both theories considered important motivational and self-regulated learning components; however, prior research falls short of addressing the effect of these components on the specific issue of learning with concept maps.

Goals can be defined as cognitive representations that serve as the focus of the transaction between one’s motivation and cognition (Ford, 1992). Achievement goals refer to the purposes or reasons for which an individual is pursuing an achievement task (Pintrich, 2000). It focuses more on understanding why individuals are trying to achieve a certain goal (Urdan & Maehr, 1995) and suggests that students’ engagement in a learning task can be motivated by a different set of goals (Dweck & Elliot, 1983; Maehr & Nicholls, 1980).

With a long tradition in educational research, studies on achievement goals have produced an array of conceptual models. The dichotomous model (Ames, 1992; Dweck & Leggett, 1988) distinguishes between two goal types: mastery (or learning/task) goals and performance (or ability/ego-involvement) goals. Mastery goals, which are centered on knowledge acquisition and mastery of tasks, relate to the learner and the knowledge and skills to be learned. Accordingly, a student may want to succeed because he/she sincerely wants to learn and understand the material and master the content and will therefore utilize higher learning strategies (Kurtz & Borkowski, 1984), display high self-efficacy and task persistence (Pintrich & DeGroot, 1990), and thus be considered an intrinsically motivated learner (Pintrich, 2000). In contrast, students who hold extrinsic performance learning goals tend to focus on how they perform relative to each other. They aspire to achieve a good grade solely to demonstrate that they are better than others. This might indicate a greater academic ability but not necessarily content mastery (Maehr & Zusho, 2009).

Approaches to learning refer to students’ deep or surface learning in a specific learning situation (Biggs et al., 2001). Deep learners strive to understand the meaning behind the material, think critically about ideas, relate topics to prior knowledge, create new arguments, and embrace critical thinking. These self-regulated learning competencies, refer to the student’s ability to use internal controls for learning, including setting their own goals and forming an awareness of current knowledge structures (Biggs, 1989). Additionally, deep learners have the ability to continue to learn for the purpose of coping with the everchanging and growing complexity of the content they study and are intrinsically motivated towards learning (Haggis, 2003). Contrary to deep learners, surface learners use unreflective approaches to their learning, do not elaborate on facts, nor interact with ideas. Their intention is simply to memorize ideas and recite material inactively (Haggis, 2003).

Entwistle (1998/2012) and Biggs (Biggs, 1993; Biggs et al., 2001) elaborated the theory of learning approaches by creating a two-factor scale of deep and surface approaches. Their studies have borne much fruit in the field of education. These studies have pointed to several personality and environment-related correlates of students’ learning approaches, such as students’ discipline-related self-concepts (Platow et al., 2013), and problematic internet use (Alt & Boniel-Nissim, 2018). However, the connections between approaches to learning and CM utilization have yet to be explored.

This Study

The literature review shows that concept maps are still underutilized in higher education and that little is known about how higher-education students’ goal setting and approaches to learning might be connected to the way they use concept maps for effective PBL. To address this gap, the overarching aims of this study were to design an intervention program (see description below) that employs CM in PBL with a designated decision-making process and to evaluate the effectiveness of using concept maps in this process for students with different motivations for learning. The intervention was piloted for one three-month-long semester in three institutions of higher education located in Israel and Austria. The following three research questions and hypotheses were examined:

() How will students perceive the effectiveness of CM for PBL with a designated decision-making process? An effort will be made to identify the most effective CM factor/s that contributed to students’ decision-making process.

() How might students’ different learning goals be connected to their utilization of CM in PBL? Previous studies (Schaal, 2010; Sun & Chen, 2016) showed how students’ motivation for learning impacts the way they utilize CM for learning. Therefore, we hypothesized that Mastery goal-oriented students would be more receptive to adopting CM and demonstrate a greater propensity for utilizing this learning strategy than their performance goal-oriented peers. More specifically, it was postulated that a mastery-approach goal setting will be more positively connected to students’ positive perceptions of CM to support their learning (H1) than a performance-approach goal setting (H2).

() How might students’ different learning approaches (deep and surface) affect their utilization of CM in PBL? Based on previous studies (Schaal, 2010; Sun & Chen, 2016) on students’ motivation for learning and CM utilization, it was hypothesized that deep learners would tend towards positive perceptions of CM utilization in PBL (H3) as opposed to surface learners who would not (H4).

Confounding variables that might affect the variables being studied e.g., institutions’ location (nationality), gender, and age were addressed to examine and control their potential effect on the research constructs. Controlling the cross-national factor allowed the examination of the associations between the research constructs over and above potential cultural differences that might have existed between the students.

Method

Participants

Data for the analysis were gathered from a total of 129 undergraduate students from three higher education institutions, which included two academic colleges located in Israel, categorized as A and B (A = 61 students; B = 41 students), and a teacher-training college in Austria (27 students). The mean age of the participants was 24.2 years (SD = 6.36), 86.9% were females, and 60% were native speakers (i.e., Hebrew in Israel and German in Austria). The students from Israel were enrolled in Education and Health study tracks, while those from Austria were enrolled in a course titled “Inclusive Education”. Table 1 presents the student characteristics for each institution. The between-college group analyses revealed significant differences in the variables of age and language (native/non-native speakers). Regarding the age variable, the Austrian teacher-training college’s sample comprised significantly younger participants (F(2) = 8.63; p < .001) and had significantly more native German speakers (F[2] = 44.53; p < .001).

Table 1

Student charecteristics by institutes

	Israeli college A(N = 61)	Israeli college B(N = 41)	Austrian college(N = 27)
Age (mean and SD)	25.70 (SD = 6.02)	24.76 (SD = 7.69)	20.87 (SD = 0.88)
Language (percentage of native speakers)	72.1%	17.1%	100%
Gender (percentage of female students)	85.2%	87.8%	92.6%

The Intervention

Based on a number of approaches identified with the constructivist approach to learning, (e.g., cognitive schemes, moral development theory, dilemma discussion, moral reasoning schemes, radical theory, and social-cultural constructivist theories), the approach to learning, known as Values and Knowledge Education (VaKE), was designed and piloted in numerous studies (e.g., Patry et al., 2013). It is considered a useful teaching tool that combines ethics and values-oriented education with content-knowledge education, with an emphasis on social behavior and the development of critical thinking in a PBL environment. In line with VaKE, the students in the present study were given a problem connected to their course content (e.g., intercultural education). For example,

Martin, Sophia and Selma are three young people from Austria, Germany and Israel. All three have recently finished their teacher training and will start their first teaching position in the upcoming school year. The summer prior to that, the three of them had the chance to participate in a weekend workshop in which an NGO brings together young teachers from different continents to explore the topic of "Cultural Diversity and School". Freshly arrived at the venue, the three attend the opening speech of a school official who describes in flowery words the opportunities afforded by cultural diversity. During the lunch break, the three young people get into a conversation. Martin is very enthusiastic and says, “I really liked what we heard. I want to make the cultural diversity in my classroom more visible and use it as a resource. I think everyone benefits from it, even the children from the majority culture. There are so many possibilities. Whenever possible in my lessons, I will involve pupils with an immigration background as experts for their cultures of origin. For that there are so many possibilities. For example, you can take into account the students' multilingualism by asking them for translations in different languages. Religiously based food regulations could be addressed in religion and ethics lessons, and so on.” Sophia made a grimace: "‘Togetherness’ is quite nice, but this [official's] speech also sounded very naive. Of course, when different people come together in an uninhabited place, you have to negotiate the rules together. But we don't have such a situation in our countries. There is the majority culture, and it should be the leading culture. In my opinion, this applies to the classroom as well. There [immigrant] children from other, outside cultures should be obligated to know the rules of their new country and its cultural customs." Selma looks at both young teachers skeptically and says, “I don't know what 'culture' is at all. Films and novels are culture; people speak of Christian, Islamic and Jewish culture. I don't want to base my lessons on such a questionable construct. For me, we are all human beings and, therefore, are all equal. Cultural background has no place in my teaching. I teach students and not representatives of any given culture. How the students want to define themselves outside of class is up to them (Dähling, 2019).

The study participants were asked, ‘Which perspective should the teachers embrace?’ The task had two phases. In Phase 1, the participants were asked to detail three arguments used to shape their decision by using a concept map and discussing it with their peers. Mindomo, a popular platform for designing concept maps was utilized. Mindomo allows the user to visually outline complex concepts, tasks, ideas, and thoughts. Mindomo enables real-time collaboration, has multiple layouts (mind map, concept map, task map, outline, etc.), and functions as a hypertext-based environment enabling the integration of a wide variety of formats.

In Phase 2, based on the materials taught in their courses, the participants were asked to obtain the necessary supporting information to substantiate their arguments and associate a relevant value with at least two of them. Finally, the participants were instructed to specify links between their arguments, explaining the differences or similarities between them. To assess their maps, well-established criteria were provided to the students, as illustrated in Table 2. This assessment tool was adapted from Panadero et al. (2013) to meet the current study’s goals. Figure 1 is an exemplary map designed by a group of Israeli students in Phase 2.

Table 2

Rubric for assessing the concept map

Criteria / score	4	3	2	1
Arguments and supporting information	All three arguments and three supporting information are included.	Two arguments and two supporting information are included.	One argument and supporting information are included.	Arguments and supporting information are lacking.
Hierarchy	The organization is complete and correct. The supporting information corroborates the arguments.	The organization is correct but incomplete. Most of the supporting information corroborates the arguments.	The organization is correct but incomplete. Most of the supporting information does not corroborate the arguments.	The organization is incomplete and/or incorrect.
Relationships among arguments / supporting information	Relationships were specified and well-explained. Links to ethical values were added and explained.	Relationships were partly specified but explained. Links to ethical values were partly added but explained.	Relationships were partly specified but not explained. Links to ethical values were partly added but not explained.	Relationships were partly or not specified and poorly/not explained. Links to ethical values were incorrect or missing.
Simplicity and easiness of understanding	The design is simple and easy to understand.	Some relationships are difficult to understand.	There is an excessive number of connections.	Neither the relationships nor the hierarchy are understandable.

Fig. 1

An exemplary map designed by a group of Israeli students

Measurements

Student Characteristics

Data were gathered using a questionnaire aimed at measuring the student’s gender, age, nationality (Israeli vs. Austrian), and local language proficiency (native/non-native speakers).

Concept Mapping for Problem-Based Learning Scale (CM-PBL)

This 12-item questionnaire, as displayed in Table 3, was designed for the purpose of the current study to capture students’ perceptions of the effectiveness of using concept maps in the decision-making process required in VaKE and to assess how it helped them during this process in relation to the problem, across four factors: cognitive aspects, affective aspects, self-regulation of learning, and transfer of learning. Participants were asked to indicate their level of agreement with each of the statements shown in Table 3. The items were scored on a six-point Likert scale ranging from 1 = strongly disagree to 6 = strongly agree. The questionnaire was administered twice, once after the first phase (α = .96), and again after the second phase (α = .97). A principal axis factoring analysis followed by a varimax rotation was used to corroborate the structural validity of the scale (item loadings > .30). The analysis solution accounted for 79.1% of the variance in the pretest and 85.5% in the posttest, yielding in each analysis the above four categories, as shown in Table 3.

Table 3

Factor loading and reliability results for Concept Mapping for PBL Scale

Item	Factor
	Affective aspects	Self-regulation of learning	Cognitive aspects	Transfer of learning
12. I enjoyed using concept mapping during the decision-making process.	.76	.33	.37
11. I liked using concept mapping to assist me in making decisions.	.70	.40	.36	.34
10. I was satisfied using concept mapping in making decisions.	.70		.42
7. I think concept mapping can be easily used in other decision-making discussions	.54	.39	.39	.36
5. Concept mapping helped me to reduce barriers when dealing with decision-making.		.84
6. Concept mapping enhanced my interest in decision-making.	.33	.71
4. Concept mapping stimulated me to learn and think independently.	.31	.61	.40
2. Concept mapping helped me identify the interrelationships among arguments	.32	.35	.82
3. Concept mapping helped me specify interrelationships among arguments.	.38	.34	.74
1. Concept mapping helped me learn about the topic	.53		.63
8. I will consider using concept mapping in other complex decision-making processes.	.45	.41	.34	.71
9. I will consider using concept mapping to make decisions in the future.	.44	.50		.62
Cronbach alpha reliability results	.94	.94	.92	.95

Bold numbers represent the indicators that belong to each factor

The Student Process Questionnaire (R-SPQ-2F)

The R-SPQ-2F (Biggs et al., 2001), consists of 20 items. The participants were asked to indicate their approaches to learning. The surface approach to learning sub-scale includes items such as: ‘I learn some things by rote, going over them again and again until I know them by heart, even if I do not fully understand them’. A deep approach to learning indicates that the student has an intrinsic interest in studying; for example, ‘I find that at times studying gives me a feeling of deep personal satisfaction’. The items were scored on a six-point Likert scale ranging from 1 = almost never true to 6 = almost always true. A principal axis factoring followed by a varimax rotation was used to corroborate the stability of the scale structure (eigenvalue >1.00; item loadings > .40). The analysis solution accounted for 41.8% of the variance and yielded the expected two categories: deep and surface approaches to learning. The computed internal consistencies (Cronbach’s alpha) for each factor, indicated sufficient reliability of the results within the factors (Surface learning, 10-item α = .87; Deep learning, 10- item α = .86).

Achievement Goal Questionnaire

In the present study, the 12-item Achievement Goal Questionnaire-Revised (Elliot & Murayama, 2008) was used. Participants were asked to indicate their degree of agreement with each item on a six-point Likert scale ranging from 1 = strongly disagree to 6 = strongly agree. Subscales and internal consistency values were: mastery-approach, for example, ‘My aim is to completely master the material presented in this class’; mastery-avoidance, for instance, ‘My aim is to avoid learning less than I possibly could’; performance-approach, for example, ‘My aim is to perform well relative to other students’; and performance-avoidance, for example, ‘My aim is to avoid doing worse than other students’. A principal axis factoring followed by a varimax rotation was used to corroborate the stability of the scale structure (eigenvalue >1.00; item loadings > .40). The analysis solution accounted for 74.9% of the variance and yielded only three categories: mastery-approach (three items, α = .89), mastery-avoidance (three items, α = .82), and performance, including the performance-approach and performance-avoidance items (six items α = .90).

Procedure

The CM-PBL scale was presented to the participants twice after each phase (1 and 2), in order to control for students’ first exposure to CM technique. The other scales were administered following the three-month, semester-long intervention. Participants’ prior consent was obtained. The questionnaires were anonymous, and the participants were assured that no identifying information about the specific courses would be processed. The study was pre-authorized by the first author’s college’s Ethics Committee.

Data Analysis

Data were analyzed using Partial Least Squares - Structural Equation Modeling (PLS-SEM; Hair et al., 2017), a method advised to be employed if the primary objective of applying structural equation modeling is the prediction of target constructs. SmartPLS 3 software was used.

Findings

Students’ Perceptions of Concept Mapping Effectiveness

To assess how students perceived the effectiveness of using CM in PBL in line with Q1, a multivariate analysis of variance was used to compare between the mean results of the CM-PBL sub-factors: cognitive aspects, affective aspects, self-regulation of learning, and transfer of learning separately in each test (pre and post). While non-significant differences between the factors were revealed in the pretest (F[3, 124] = 1.14, p > .05; η2 = 0.03), significant differences were found in the posttest (F[3, 119] = 22.46, p < .001; η2 = 0.36) between cognitive aspects (M = 4.27, SD = 1.26) and each of the other sub-factors: affective aspects (M = 3.75, SD = 1.35); self-regulation of learning (M = 3.83, SD = 1.30); and transfer of learning (M = 3.74, SD = 1.41).

A multivariate analysis of variance was also used to detect differences between the tests on the four CM-PBL sub-factors. The analysis yielded a significant result (F[3, 245] = 7.49, p < .001; η2 = 0.08). However, only the cognitive aspects factor was found higher at the posttest (M = 4.27, SD = 1.26) compared with the pretest (M = 3.77, SD = 1.08, F[1, 248] = 11.15, p < .001; η2 = 0.04).

Achievement Goals and Concept Mapping

To check the set of connections between students’ achievement goals and CM-PBL factor (H1 and H2), Model 1 (Fig. 2) was designed. The model includes three main latent constructs: on the left, the mastery and performance goals as independent factors, and on the right, the CM-PBL factor, each accompanied by its indicators. Relationships between the constructs as well as between the constructs and their assigned indicators are displayed as arrows. In PLS-SEM, single-headed arrows, as shown between the constructs, are considered predictive relationships, and with strong theoretical support, can be construed as causal relationships. Paths were specified from the two achievement-goal factors to the CM-PBL dependent factor. The PLS-SEM analysis used a path weighting scheme and a mean value replacement for missing values. It should be noted that based on a previously conducted analysis, background variables were also entered into the model to control their effect on the latent variables (age, gender, first exposure to CM at the pretest, local language proficiency, and nationality). Two variables were found significantly connected to the model’s constructs and were therefore included in the model: Language (1 = native speakers, 2 = non-native speakers), and nationality (1 = Israeli colleges, 2 = Austrian college).

Fig. 2

Model 1. Analysis results of the examination of H1 and H2 by SmartPLS. Note: Concept mapping for problem-based learning (CM-PBL)

To test the direct effects, the bootstrap routine was employed. As can be learned from Model 1, the mastery goals factor was found positively connected to the CM-PBL factor (β = .21 p < .01), whereas a non-significant coefficient result was detected between the performance factor and the dependent factor. The first and second hypotheses were confirmed.

Model Evaluation

The model evaluation included several steps. First, collinearity was examined by the Variance Inflation Factor (VIF) values of all sets of predictor constructs in the structural model. The results showed that the VIF values of all combinations of endogenous and exogenous constructs were below the threshold of 5 (Hair Jr et al., 2017) ranging from 1.21 to 1.48. Next, the coefficient of determination (R) value was examined, when R values of 0.75, 0.50, or 0.25 for endogenous latent variables can be described respectively as substantial, moderate, or weak (Hair Jr et al., 2017). R for CM-PBL was found weak (0.21). An even lower result was found for mastery goals (R = .17). In addition to measuring the R values, the change in the R value when a specified exogenous construct is omitted from the model was used to evaluate its impact on the endogenous constructs. This measure is referred to as the f effect size when values of 0.02, 0.15, and 0.35, respectively, represent small, medium, and large effects (Cohen, 1988). The f effect size result was 0.04 for mastery goals – CM-PBL. The f effect size results for the background variables ranged from 0.04 to 0.09. Finally, the blindfolding procedure was used to assess the predictive relevance (Q) of the path model. Values higher than 0 suggest that the model has predictive relevance for a certain endogenous construct (Hair Jr et al., 2017). The Q value of CM-PBL was 0.15.

To evaluate which of the mastery goal approaches (mastery approach or mastery-avoidance) will be more dominant in predicting students’ perceptions of the effectiveness of using concept maps in the decision-making process required in VaKE, Model 2 (Fig. 3) was designed. As can be seen in Fig. 3, the model includes the same latent constructs as in Model 1, excluding the performance goals variables. Moreover, the mastery goals variable was entered into the model by using its two sub-constructs: mastery approach and mastery-avoidance. Paths were specified from the mastery goals independent sub-factors to the CM-PBL dependent factor. As shown in Model 2, only the mastery-approach sub-factor was found positively connected to CM-PBL (β = .26 p < .001), whereas a non-significant connection was shown between mastery-avoidance and the dependent variable.

Fig. 3

Model 2. Further analysis results of the examination of H1 by SmartPLS. Note: Concept mapping for problem-based learning (CM-PBL)

Collinearity examination yielded sufficient results (inner VIF values ranged from 1.21 to 1.32). As indicated in Model 2, R for CM-PBL was 0.21. According to the f effect size results, a medium effect (0.06) was exerted by the mastery-approach on CM-PBL. Finally, the Q value for CM-PBL was 0.15.

Deep and Surface Learning Approaches and Concept Mapping

According to research hypotheses H3 and H4, deep learning was expected to increase students’ positive perceptions of CM as opposed to surface learning, which was not. To evaluate these hypotheses, Model 3 (Fig. 4) was designed. As can be seen in Fig. 4, the model includes the deep and surface learning latent constructs and the CM-PBL dependent variable. Background variables were also entered into the model to control their effect on the latent variables (age, gender, first exposure to CM at the pretest, local language proficiency, and nationality). For visual clarity, background variables with significant coefficient results are shown. As can be learned from Fig. 4, gender was found positively related to surface learning and was therefore included in this analysis. As shown in Model 3, only the deep learning sub-factor was found positively connected to CM-PBL (β = .33 p < .001), whereas a non-significant connection was shown between surface learning and the dependent variable. Hypotheses H3 and H4 were confirmed.

Fig. 4

Model 3. Analysis results of the examination of H3 and H4 by SmartPLS. Note: Concept mapping for problem-based learning (CM-PBL)

Collinearity examination yielded sufficient results (inner VIF values ranged from 1.00 to 2.25). R for CM-PBL was 0.25. According to the f effect size results, a medium effect (0.13) was exerted by deep learning on CM-PBL. Finally, the Q value for CM-PBL was 0.17.

Summary of Findings

To assess the effect of deep learning and mastery-approach on the four sub-factors of CM-PBL, Model 4 (Fig. 5) was designed. It included the independent factors that were found significantly related to CM-PBL, and the four CM-PBL sub-factors of cognitive aspects, affective aspects, self-regulation of learning, and transfer of learning. Paths were specified between the independent and dependent variables; however, for clarity purposes, only the significant paths are shown in Fig. 5.

Fig. 5

Model 4. Analysis results with Concept Mapping for PBL sub-factors by SmartPLS. Note: Concept mapping for problem-based learning (CM-PBL)

According to the findings, mastery approach significantly contributed to self-regulation of learning (β = .17 p < .05), whereas non-significant connections were shown between this variable and the other dependent variables. Deep learning was found significantly related to all the dependent variables (p < .001), with similar (moderate) coefficient path results. Collectively, the independent variables explained 27% of the self-regulation of learning variance. Hence, deep learners, who aim to completely master the material presented in this class, perceived CM for PBL as a largely effective tool for self-regulating their learning during the intervention. These results seem stable also when controlling for background variables such as nationality and age. Yet it should be noted that Austrian students gained lower scores on mastery-approach and perception of CM for PBL than their Israeli counterparts.

Collinearity examination yielded sufficient results (inner VIF values ranged from 1.07 to 2.07). R values for CM sub-factors are shown in Fig. 5. According to the f effect size results, a low effect (0.05 – 0.10) was exerted by deep learning on CM sub-factors. Finally, the Q value for CM sub-factors ranged from 0.15 (affective aspects) to 0.22 (self-regulation of learning).

Discussion

The main aim of this study was to design an intervention activity that employs digital concept mapping in problem-based learning within a designated decision-making process. The activity was based on the VaKE constructivist approach to learning. Another aim was to evaluate the effectiveness of using concept maps in this activity for students with different motivations for learning. This study mainly revealed that the intervention can benefit higher education, above and beyond their nationality, and study track affiliations, specifically at the cognitive level, to identify interrelationships among arguments and learn about the topic. These cognitive abilities can be construed as conceptual domain-specific knowledge (Anderson et al., 2001) referring to the knowledge of concepts, classifications and categories, principles, generalizations, theories, models, and structures, reflecting a deep understanding of content (Biggs et al., 2001). This set of abilities places an emphasis on how knowledge fits into larger systematic perspectives (Wilson, 2016).

Another finding showed that deep, mastery-approach learners will take this domain-specific knowledge another step forward towards self-regulation. Hence, deep learners, who strive to completely master the material presented in class would be likely to perceive the CM for PBL as a largely effective tool for self-regulating their learning during the intervention. This ability is considered metacognitive knowledge of general strategies for learning and thinking. The student thinks about his/her strategic planning and control of these processes (Stanton et al., 2015). Weinberger and Fischer (2006) maintained that these types of knowledge can be achieved by encouraging students to build sequences of arguments that represent a knowledge-building cycle. Knowledge building in this context requires that learners construct arguments to justify their position, as practiced in the designated intervention in the current study.

Regarding the confounding variable measures, the analysis mainly showed that the Austrian students were less receptive to adopting CM and were less oriented towards mastery goals than Israeli students. A plausible explanation can be related to the Austrian students being younger than their Israeli peers, and to the empirical model additional results showing that mature-age students are more likely to adopt a deep learning approach, as also suggested by previous studies (Lake & Boyd, 2015). Thus, one possible explanation is that dealing with tasks of high difficulty might cause younger individuals to struggle and thereby incur surface learning approaches (Zhang et al., 2016).

Strength, Limitations, and Mitigations

The results of this study essentially expand upon empirical results of previous studies by focusing specifically on higher education students and by indicating for the first time that learners with specific motivational proclivities, particularly those who focus on mastery- approach goals and deep learning, will be inclined to utilize concept maps in PBL to support their self-regulation of learning. Nonetheless, the study presented herein has some methodological limitations. A newly developed scale was used to measure how the research participants perceived digital CM as a tool to foster learning, yielding satisfactory validity and reliability results. Nevertheless, insofar as this scale is based on self-reports, the data are liable to be biased to some extent. To further establish the validity of our findings, it might be fruitful to additionally assess the quality of the concept maps as learning products in the PBL setting and compare the results to the self-reported findings.

Another limitation of the study is the lack of measurement of a collaboration factor in the CM-PBL questionnaire, even though the concept maps during the intervention were in fact produced collaboratively. It might be valuable to add items such as: ‘the CM platform helped me share my work’ or ‘collaborate with others’ so as to capture and measure the collaboration aspect of the intervention. Moreover, research on collaborative learning shows that discussions in a learning environment may differ in quality. Sometimes in group learning settings, the students may “just talk” without reacting to the others´ contributions (cumulative talk), or they may share their own viewpoint but remain with their divergent perspectives and not reach a deeper level of understanding each other (disputational talk). Or in other cases, they may really argue substantively and reach the highest level of collaboration in problem-solving (explanatory talk, Wegerif et al., 1999; Wuttke, 2012). A fruitful avenue for future studies might be to explore the quality of the discussions during an online activity incorporating CM with PBL. For example, it would be interesting to shed light on how teachers could support students within VaKE to attain the level of explanatory talk and how the use of concept maps may aid in fostering an effective argumentation-based discussion among learners.

In the present study, a digital tool for CM was used, yet this tool was not compared with a non-digital activity. To mitigate this limitation, it might be interesting to compare these practices and assess the connection of each practice in relation to the factors examined in the current study. For example, researchers should evaluate the plausible hypothesis suggesting that for some students it might be overwhelming to deal simultaneously with a CM method applied on a newly introduced digital tool to solve (complex) problems, compounded by an online setting that demands self-regulated and collaborative learning. (Kirschner et al., 2009). Qualitative data might mitigate this limitation, by asking the students to narrate their experiences via open-ended questions, interviews, or reflective journals. Moreover, because the present study did not provide insights into the process of how concept maps support deep and self-regulated learning and how different groups of learners benefit from using concept maps, it would be fruitful to gather process data via continuous state sampling such as motivational and emotional states (Kärner et al., 2017; Kögler & Göllner, 2018).

Conclusions, Implications, and Directions for Future Studies

Despite its limitations, the results indicate that using concept maps in PBL online settings may have positive effects on learning, in line with previous research results regarding concept maps for constructivist PBL (e.g., Chan, 2017) and individual differences (Schaal, 2010; Sun & Chen, 2016). The VaKE approach used in this study, as a special constructivist approach in the context of values-oriented education and ethical thinking, considers self-regulation or internal control for learning as a core requirement. Moreover, the central methodology within VaKE is the discussion of a conflict of values that can, in turn, foster moral reasoning (Weinberger et al., 2016). The current study elaborates on previous studies by showing how VaKE session discussions centering on moral conflicts might be supported by using digital concept maps and encouraging the learners to exchange different arguments. By organizing and structuring the information and the relationships between them via concept maps the participants may also gain knowledge and learn much more about the topic.

The current study suggests that mastery-approach students and deep learners might be more intrinsically motivated and use superior learning strategies, such as concept maps, towards internal control for learning. Therefore, teachers should carefully examine how CM can be optimally introduced to different types of learners. It might be advisable to scaffold constructivist activities that incorporate concept maps designed for surface learners so as to avoid overly strenuous demands and thus reduce ineffective use of, or engagement with, the activity. Indeed, high-order thinking skills, deep learning, and technological skills are deemed essential in the current conceptual era; however, some people may struggle to sustain current digital practices and may need to ‘slow things down’ (Rauch, 2018). Thus, it might be an exciting new direction for further research to evaluate creative ways that might scaffold surface learners’ engagement in activities that enable them to solve complex problems. This can be achieved by pointing out the benefits of technology-enabled platforms such as digital CM activities for their learning and assessing whether such interventions would lead them to recognize CM as a practice that fosters meaningful learning.

The results presented herein initially illustrate that students’ goal settings and approaches to learning are important criteria in the overall evaluation of CM pedagogical technique. However, this study also indicates that there is still room for improvement in the implementation of CM. It might be inferred that merely infusing technology into a learning activity is insufficient. Teachers should encourage students to recognize the benefits of the technological tool for their individual learning needs.