The influence of thinking dispositions on integration and recall of multiple texts.

We investigated the association between thinking dispositions and two outcomes of multiple-texts comprehension: integration of conflicting information in argumentative essays; and recall of inferential information as an index of deep comprehension. We focused on two thinking dispositions, need for cognition (NFC) and actively open-minded thinking (AOT), as relevant individual differences in the processes involved in multiple-texts comprehension. NFC is the tendency to engage in and enjoy cognitively demanding activities, whereas AOT is the tendency to rationally evaluate arguments and supporting evidence, without being influenced by biases from one's own prior beliefs and prior knowledge. 73 university students completed perceived topic knowledge, perceived exposure to argumentative writing, and perceived competence in argumentative writing, NFC and AOT questionnaires, read two contradictory texts, wrote an argumentative essay, and recalled the information read 1 month later. Argumentative essays were assessed by length and level of integration of conflicting perspective. Text recalls were assessed by number of valid inferences included. Research questions were investigated through a path analysis model. The path analysis model had a good fit. NFC was indirectly associated with argumentation quality of the essay via the essay length. AOT was directly associated with the inferences included in the recall task. The present study contributes to the literature on multiple-texts comprehension by emphasizing the role of thinking dispositions.

Background

In the present‐day knowledge society, to participate in the democratic discourse, people need to critically comprehend and integrate information across multiple sources that express diverse and contradictory viewpoints. However, the effortful processing of multiple documents is not always effective, even among adults, as it depends on readers' skills and dispositions (Bråten, Britt, Strømsø, & Rouet, 2011). Readers often struggle in integrating information from alternative perspectives, fail in evaluating the plausibility of arguments, and construct one‐sided representations (Mateos et al., 2018). Research on multiple‐texts comprehension has examined the role of several cognitive factors, such as prior knowledge (e.g., Strømsø, Bråten, & Britt, 2010) and sourcing skills (Strømsø, Bråten, Britt, & Ferguson, 2013). Conversely, more research is needed to investigate the dispositional factors that promote students' competence in integrating conflicting information when involved with multiple‐source inquiry activities. Prior research has emphasized the role of the disposition to engage in and enjoy activities that require thinking (i.e., need for cognition, NFC) on multiple‐texts comprehension, but limiting the analysis to one outcome at a time (see Bråten, Anmarkrud, Brandmo, & Strømsø, 2014; Kardash & Scholes, 1996; Tarchi & Mason, 2020; Winter & Krämer, 2012). Even fewer studies have investigated the contribution of readers’ disposition to weigh new evidence against a favoured belief heavily (i.e., actively open‐minded thinking, AOT; see Griffin, Wiley, Britt, & Salas, 2012; Stanovich & West, 1997). To advance current research on multiple‐texts comprehension, we investigated the association between thinking dispositions – NFC (Cacioppo & Petty, 1982) and AOT (Stanovich & West, 1997) – and two outcomes of multiple‐texts comprehension: (1) integration of conflicting information in argumentative essays; and (2) recall of literal and inferential information, the last one representing an index of elaboration and deep comprehension (Diakidoy, Christodoulou, Floros, Iordanou, & Kargopoulos, 2015).

Thinking dispositions

Thinking dispositions are cognitive styles, ‘habits of mind’ (Heijltjes, van Gog, Leppink, & Paas, 2015; Stanovich, West, & Toplak, 2016), that reflect the individual's opinions, beliefs, and reflectively acquired goal structure (Ganuthula & Dyaram, 2016; Stanovich, West, & Toplak, 2011). Thinking dispositions provide clues to which mechanisms are involved in suboptimal thinking (Stanovich et al., 2016). In the present study, we will focus on two thinking dispositions, NFC and AOT, as relevant individual differences in the processes involved in multiple‐texts comprehension.

NFC is the tendency to engage in and enjoy cognitively demanding activities (Cacioppo & Petty, 1982). High levels of NFC are associated with intrinsic motivation to use cognitive skills and rely less on the heuristics of the algorithmic mind (Ganuthula & Dyaram, 2016). When associated with reading comprehension, NFC was defined as the disposition to engage in a deep understanding of the text, and enjoy the process of a cognitively demanding activity that leads to the construction of a well‐integrated situation model (Dai & Wang, 2007).

AOT is the tendency to rationally evaluate arguments and supporting evidence, without being influenced by biases from one's own prior beliefs and prior knowledge (Mellers et al., 2015). On the one hand, prior knowledge can be very helpful for their learning, especially when students are able to connect new information to it (Tarchi, 2010; McNamara & Kintsch, 2009). On the other hand, laypeople have a tendency to overestimate their expertise in a certain topic (see also the Dunning–Kruger effect or the ‘illusion of knowledge’ effect or the ‘illusion of explanatory depth’; Rozenblit & Keil, 2002). In these cases, AOT may counteract the effect of these biases, especially in those scenarios in which texts represent conflictual but equally valid perspectives on a topic. AOT is inspired by the work of Baron (1985) and includes constructs such as avoidance of considering knowledge as absolute and certain (i.e., epistemological absolutism), willingness to perspective switch, willingness to decontextualize, and tendency to consider alternative opinions and evidence (Stanovich et al., 2016). Overall, AOT consists of two processes: (1) increased seeking of counterattitudinal information and (2) higher levels of active processing of such information (Stanovich & West, 1997; Stenhouse et al., 2018). AOT is ‘open’ as it allows consideration of new possibilities, goals, and evidence, and is ‘active’ as it seeks new possibilities, rather than waiting for them (Baron, 2008). In this last respect, AOT is similar to NFC, although AOT is specifically associated with a higher elaboration of counterattitudinal evidence, rather than higher elaboration in general (Stenhouse et al., 2018).

Multiple‐texts comprehension: Definition and outcomes

Multiple‐texts comprehension involves the ‘building of a coherent mental representation of an issue from the contents of multiple documents that deal with the same issue from different perspectives’ (Bråten, Ferguson, Anmarkrud, & Strømsø, 2013, pp. 322–23). When reading multiple texts, firstly readers need to represent the internal meaning of each text (i.e., textbase) and link it to relevant prior knowledge (i.e., situation model). Secondly, readers need to create an internal representation that integrates content across texts, including agreements and discrepancies in the accounts they read (i.e., situations model or integrated mental model; see Documents Model, Britt, Perfetti, Sandak, & Rouet, 1999; Perfetti, Rouet, & Britt, 1999). In the present study, we will investigate two outcomes of multiple‐texts comprehension: argumentative essay writing and text recall.

Argumentative essay

A way to ascertain whether students construct an appropriate integrated mental model is by analysing how they represent their view in their written productions, for instance, in an argumentative essay (Barzilai & Eshet‐Alkalai, 2015; Barzilai, Zohar, & Mor‐Hagani, 2018; Diakidoy, Ioannou, & Christodoulou, 2017; Primor & Katzir, 2018). Writing an argumentative essay is a complex assignment because students have to: (1) understand the ideas presented in the source texts; (2) select relevant information; (3) organize, compare, and integrate the selected ideas, both within each text (intratextual) and between texts (intertextual integration); and (4) create a new structure to present their point of view in writing (Barzilai & Eshet‐Alkalai, 2015; Segev‐Miller, 2004).

Intertextual integration, that is, going beyond the perspectives presented in the texts and elaborating a coherent approach, is the core element when writing from contradictory texts (De La Paz & Felton, 2010; Kobayashi, 2015; Mateos et al., 2011, 2018). In an argumentative essay, intertextual integration implies being aware of the arguments supporting the different perspectives involved in the controversy and trying to solve the dispute (Barzilai & Eshet‐Alkalai, 2015). The writer may employ a variety of strategies when dealing with the different viewpoints, which vary in the degree of integration of arguments and counterarguments (Mateos et al., 2018; Nussbaum, 2008; Nussbaum & Edwards, 2011). Refutation is a low‐integration strategy that defends one perspective, by identifying its supporting arguments and considering the arguments supporting the other perspective as wrong, irrelevant o unsatisfactory (Barzilai & Weinstock, 2020; Nussbaum, 2008; Nussbaum & Edwards, 2011). Nevertheless, refutation can be used in the context of balanced reasoning if used selectively when readers come across inaccurate, poorly supported, or even fake information (Kendeou, Walsh, Smith, & O’Brien, 2014; Larson, Britt, & Kurby, 2009; Nussbaum, 2008). Conversely, when two or more perspectives are represented in the text, equally valid or at least each valid to a certain extent, weighting and synthesizing should be applied (Felton, Crowell, & Liu, 2015; Mateos et al., 2018). These are high‐integration strategies, as they involve a critical evaluation of the two viewpoints, accepting, at least partially, arguments supporting both. When weighting, a writer considers the pros and cons of both perspectives and finally identifies the stronger position. When synthesizing, the author identifies an intermediate position between the conflicting perspectives, which retains advantages (identification of better arguments) and minimizes disadvantages (acceptance of counterarguments). These two strategies are linked to better reading comprehension and deeper learning (Felton et al., 2015; Mateos et al., 2018). Refutation, weighting, and synthesizing represent the three main integration strategies explored in literature, although they have been described sometimes with different terms. For instance, synthesizing has been defined as a strategy in which writers design a solution that maximizes advantages while minimizing disadvantages of an alternative (‘constructing a design claim’; Shehab & Nussbaum, 2015). Moreover, synthesizing can be associated with what Felton and colleagues define as overcoming myside bias through consensus‐seeking dialogue (Felton et al., 2015). When texts offer valid arguments supporting multiple perspectives, readers should attempt synthesizing, although this is a resource‐consuming strategy.

A recurring theme in the literature on multiple‐texts comprehension and argumentative writing is the importance for readers and writers to decouple their prior beliefs from the evaluation of evidence and arguments (Macpherson & Stanovich, 2007; Richter & Maier, 2018; Stanovich & West, 2007). However, the literature also suggests that individuals have difficulties in this process. Overall, people use their prior beliefs (as in the myside bias) or prior knowledge (as in the belief bias) to evaluate arguments and reasons (Macpherson & Stanovich, 2007). These difficulties extend to argumentative writing too, as argumentative essays tend to be characterized by myside bias (Perkins, Faraday, & Bushey, 1991) or one‐side reasoning (Nussbaum, 2008), which refers to not taking into account the other perspective’s argumentations neither incorporate them in one’s discourse. Overcoming these biases and achieving two‐side reasoning is fundamental as it leads to deep learning (Felton et al., 2015; Kobayashi, 2015; Mateos et al., 2018; Voss & Van Dyke, 2001). A higher engagement with the material can induce readers to take into consideration also belief‐inconsistent sources and, consequently, achieve two‐side reasoning. The essay length (i.e., the number of words included in the essay) is considered as an index of engagement (Eisenberger, Masterson, & McDermitt, 1982; Latini, Bråten, Anmarkrud, & Salmerón, 2019) and thus can be hypothesized to be a mediator of argumentation quality (MacArthur, Jennings, & Philippakos, 2019).

Text recall

Although the ability to evaluate and to integrate arguments included in texts about controversial issues is fundamental for students' learning and decision‐making processes, some studies suggest that it may be dissociated from long‐term recall. In two separate studies, Diakidoy et al. (2015, 2017) investigated the relationship between the critical evaluation of arguments and text recall. In specific, they investigated inference generation and main claim recall as indicative of the extent to which the text was processed deeply. They found that neither of these indices of deep comprehension contributed to the critical evaluation of arguments, a result that suggests that the inferences generated may mainly represent local connections or argument‐irrelevant elaborations. Whereas argumentative evaluation (and writing) is essentially rational processes, comprehension may rely on automatic processing of text information, unless the reader decides to implement deliberate elaboration processes. Indeed, inferences may be irrelevant if available prior knowledge is inaccurate (Kendeou & Broek, 2005). Thus, besides investigating the quality of the argumentative essay, it is also important to evaluate students' recall of argumentative texts. Forming an accurate representation in short argument scenarios may seem relatively straightforward, but it could become overwhelming when engaged with longer argumentative texts, multiple texts, or when readers are asked to recall information read in the past (Diakidoy et al., 2017).

Thinking dispositions and multiple‐texts comprehension

A few studies have explored the association between thinking dispositions and multiple‐texts comprehension, mainly focusing on NFC. NFC was found to be associated with a stronger preference for selecting two‐sided over one‐sided articles (Winter & Krämer, 2012). NFC was found associated with undergraduate students' ability to write texts in which opposing perspectives are integrated after reading texts about a controversial issue (Kardash & Scholes, 1996). Finally, NFC was found indirectly associated with multiple‐texts comprehension through the mediation of deeper‐level strategies (Bråten et al., 2014).

Despite no previous study has investigated the effect of AOT on multiple‐texts comprehension, a few indications can be derived from studies with similar variables involved. Griffin et al. (2012) administered a partial version of AOT, focused on assessing commitment to logic, evidence, and reasoning to seventh‐grade children, and found that AOT was associated with learning performances in a multiple‐document inquiry task in science. Stanovich and West (1997) found a positive association between AOT and an argument evaluation task. Both studies suggest the existence of an association between AOT and multiple‐texts comprehension; however, Griffin et al.’s study (2012) included children as participants and assessed evidence‐based thinking, rather than AOT, and Stanovich and West (1997) asked to evaluate arguments, rather than integrate conflicting perspectives, and presented scenarios, rather than actual texts.

The present study

The present study investigated the association between two thinking dispositions, namely NFC and AOT, and multiple‐texts comprehension, as assessed through an argumentative essay and a recall task. For the argumentative essay, we included in the statistical analyses the argumentative quality (Barzilai & Eshet‐Alkalai, 2015; Barzilai et al., 2018; Diakidoy et al., 2017; Primor & Katzir, 2018) as an outcome variable and essay length as a mediator (MacArthur et al., 2019). For the recall task, we measured valid inferences generated as an index of the quality of long‐term text recall (see Diakidoy et al., 2015). Prior topic beliefs, perceived topic knowledge, perceived exposure to argumentative writing, and perceived competence in argumentative writing were included as control variables.

The analysis of the literature suggests that AOT and NFC should be involved in both multiple‐texts tasks, as they should foster a deeper elaboration of texts and a more integrated situation model (Dai & Wang, 2007), especially when texts present different perspectives on a controversial topic (Stanovich et al., 2016). Conversely, some studies suggest that there may be a dissociation between different tasks involved in argumentative texts reading (Diakidoy et al., 2015, 2017), depending on readers' level of elaboration of the texts. Such dissociation may influence the role played by AOT and NFC. Consistently with the literature on multiple‐texts comprehension and argumentative writing, we expect an association between perceived competence in the task (perceived topic knowledge, perceived exposure to instruction in argumentative writing, and perceived competence in argumentative writing) and prior topic beliefs with multiple‐texts outcomes. The literature review suggests that AOT should moderate the strength of individuals’ prior beliefs and their effects on cognitive tasks (Stanovich & West, 1997; Stanovich et al., 2016) and that prior beliefs have a (detrimental) effect on multiple‐texts comprehension and argumentative reasoning (Macpherson & Stanovich, 2007; Richter & Maier, 2018; Stanovich & West, 2007; see Figure 1).

Figure 1

Expected (parsimonious) model (NFC = need for cognition; AOT = actively open‐minded thinking).

Method

Participants

Seventy‐three university undergraduate students participated in the study (age = 21.74 ± 3.76, 69 females). They were enrolled in a Psychology course, offered within the School of Education curriculum, in a large university in Italy. Students participated voluntarily and received a bonus for their participation. All participants were Italian and spoke Italian as their primary language. The sample was relatively homogeneous (i.e., middle class) regarding socioeconomic status, as assessed through parents’ occupation. Informed consent was obtained from all participants. The study followed all the indications of the Declaration of Helsinki (World Medical Association, 2013) and was approved by the Ethics Committee of the University of Florence (Italy).

Procedure

The present study is part of a larger project focusing on improving students' multiple‐texts comprehension and argumentative writing skills. Data were gathered over 2 months in four steps. First, students' self‐report questionnaires were administered. Second, students were assigned two texts about the evaluation of teachers (one pro, one against), with the following instructions: ‘Please, read these two texts discussing two different positions on a controversial topic in Education. Your task is to write an argumentative essay in which you discuss your perspective, taking into consideration what you have read’. Texts were presented on screen. Texts order was randomly varied among the participants. Third, immediately after reading the texts, students were asked to write an argumentative essay reporting their stance. Finally, a month after, students were asked to recall what they had read in the texts. Texts were available while writing the essay but not when recalling.

Materials and measures

Texts

The two texts used in this study were derived from Mateos et al. (2018) and translated from Spanish into Italian by a native speaker. The texts were similar for length (640 for the pro‐evaluation text and 769 for the against‐evaluation text1). The texts were balanced for difficulty (Gulpease index, Lucisano & Piemontese, 1988) of 42 for the pro‐evaluation text and 46 for the against‐evaluation text; thus, both are appropriate for undergraduate students. Both texts included nine arguments, and the supporting reasons were interconnected, so arguments of one position could be considered as counterarguments for the other position. For example, the pro‐evaluation text claimed that parents should participate in the evaluation of teachers and the against‐evaluation text claimed that families are not reliable sources to be used to evaluate teachers because they are not professionals.

Perceived competence

Perceived competence was assessed through three items: perceived topic knowledge, perceived exposure to instructions in argumentative writing, and perceived competence in argumentative writing (α = .72). Students were asked to report their perceived topic knowledge (‘what is your level of knowledge of the topic “the evaluation of teachers”?’) on a 6‐point Likert scale (1 = minimum; 6 = maximum). Perceived topic knowledge can be considered as a proxy for topic knowledge (Andiliou, Ramsay, Murphy, & Fast, 2012; Stanovich & West, 2008) and has been used in prior studies on multiple‐texts comprehension (Bråten, Salmerón, & Strømsø, 2016). Students were asked to report their exposure to instruction in argumentative essay writing (‘How much instruction have you received on writing argumentative essays?’) on a 6‐point Likert scale (1 = minimum; 6 = maximum). Students were asked to report their perceived competence in argumentative writing (‘How much competent are you in writing argumentative essays?’) on a 6‐point Likert scale (1 = minimum; 6 = maximum).

Prior topic beliefs

Students' prior topic beliefs were collected through an eight‐item self‐report questionnaire on a 6‐point Likert scale (1 = completely disagree; 6 = completely agree; α = .71). Four items were pro‐evaluation, and four items were against‐evaluations. An example of an item was: ‘Students' outcomes cannot be simply attributed to the influence of a single professor’. An analysis of data showed that the domain stance was the pro‐evaluation one. Thus, we proceed by reversing against‐evaluation items and calculating a total score of pro‐evaluation prior topic beliefs. A high score indicated agreement with the pro‐evaluation position, and a low score indicated agreement with the against‐evaluation position.

Need for cognition

Students' NFC was assessed through an Italian translation of the original 18‐item instrument developed by Cacioppo and Petty (1982). Each item (e.g., ‘I would prefer complex to simple problems’) was rated on a 5‐point Likert scale (1 = absolutely false; 5 = absolutely true; α = .86).

Actively open‐minded thinking

Students' AOT was assessed through an Italian translation of the original 41‐item instrument developed by Stanovich and West (1997). Each item was rated on a 6‐point Likert scale (1 = strongly disagree; 6 = strongly agree). AOT includes seven subscales, but as suggested by the authors of the instrument, a composite AOT score was formed by summing the scores on the Flexible Thinking (e.g., ‘If I think longer about a problem I will be more likely to solve it’), Openness‐Ideas (e.g., ‘I have a lot of intellectual curiosity’), and Openness‐Values (e.g., ‘I believe that laws and social policies should change to reflect the needs of a changing world’) scales and summing the sums of the reversed scores of the Absolutism (e.g., ‘Right and wrong never change’), Dogmatism (e.g., ‘There are two kinds of people in this world: those who are for the truth and those who are against the truth’), and Categorical Thinking (e.g., ‘I tend to classify people as either for me or against me’) scales (α = .79).

Students were asked to write an argumentative essay reporting their stance. A research assistant coded the essays written by the students following the coding system developed by Mateos et al. (2018). The research assistant was specifically trained by one of the authors of the coding system. The essays were first analysed to identify the number of claims from both texts and then coded on six progressive levels of integration, taking into account the type of claims and the type of conclusion. The following scores were assigned:

0 points) no sources‐based (opinion not based on the arguments presented on the source texts);

1 point) neutral position (a clear standpoint is not identified because arguments of the two positions are presented but not integrated);

2 points) supporting a position (one of the positions is defended, basically considering its arguments and not the other view);

3 points) integration of two positions via refutations (reasons for both positions are considered but the opposite view’s arguments are just refuted);

4 points) minimal integration via weighting or synthesis (one position is defended, but at least two arguments of the other view are valued, and the conclusion is partial);

5 points) partial integration via weighting or synthesis (one position or both are claimed, including arguments of both views integrated but the conclusion is missing or partial);

6 points) full integration via weighting or synthesis (the essay concludes with a real overall conclusion, considering several arguments of both positions integrated). The second author of the present study acted as an independent rater and coded 50% of the essays, which were randomly selected. The inter‐rater agreement was k = .78.

Recall task

One month after reading the texts, students were asked to recall what they had read (without access to the texts). A research assistant, specifically trained by one of the authors of the present study, coded the recall protocols written by the students following the coding system developed by Diakidoy et al. (2015). Recall protocols were parsed into clauses (k = .93), and each clause was identified either as an explicit idea recalled from the texts (verbatim or paraphrase) or as a valid inference. The outcome variable was the number of valid inferential clauses as we were specifically interested in the depth of comprehension. We also counted the total number of clauses. The first author of the present study acted as an independent rater and coded 50% of the essays, randomly selected (k = .79).

Results

Descriptive statistics are reported in Table 1. Overall, students were pro‐evaluation (scores could range between 8 and 48, with a median of 31) and reported an average knowledge of the topic (score could range between 1 and 5, mean = 2.89 ± 1.1) and some exposure to instruction (range = 1–5, mean = 3.30 ± 1.13) in argumentative writing. Students wrote essays with poor argumentative quality (mean between score 2 and 3, with the majority of students [56%] displaying a neutral position or arguing towards one position only). Performances in the recall task were low: Students were able to recall less than two inferential clauses from each text. Correlational analyses are reported in Table 2. Essay length positively correlated with the perceived knowledge of the topic, but it negatively correlated with perceived competence in written argumentation. This last result may be an indirect confirmation of the participants’ tendency towards the overestimation of their competence (Rozenblit & Keil, 2002). Importantly, the argumentative essay quality positively correlated with essay length. While writing a longer essay may be considered as a sign of low quality, for instance, when many words are used to express a simple point, the positive association with the argumentative quality suggests that it should rather be considered as a measure for engagement, in line with prior studies (Eisenberger et al., 1982; Latini et al., 2019; MacArthur et al., 2019). Recall of inferential clauses positively correlated with AOT. AOT positively correlated with prior topic beliefs, but negatively correlated with perceived knowledge of the topic. Perceived knowledge of the topic, perceived competence in written argumentation, and perceived instructions in argumentation were intercorrelated.

Table 1

Descriptive statistics (n = 73)

Variable	Minimum	Maximum	Mean	SD
AOT	130	203	175.18	17.33
NFC	37	77	59.79	9.42
Perceived knowledge of topic	1	5	2.89	1.10
Perceived competence in written argumentation	1	5	2.70	1.01
Perceived instruction in argumentation	1	5	3.30	1.13
Prior beliefs	23	39	31.64	3.12
Essay – argument quality	1	6	2.75	1.34
Essay length	129	1400	502.03	241.452
Recall – inferential clauses	1	5	1.74	1.05
Recall – total clauses	2	18	8.34	3.58

Table 2

Intercorrelations between the examined variables (n = 73)

		1	2	3	4	5	6	7	8	9	10
1	AOT	1	.12	−.37**	−.17	−.002	.24*	.08	.20	.31*	.09
2	NFC		1	.01	.05	.17	.11	.05	.33**	.17	.05
3	Perceived knowledge of topic			1	.62**	.34**	−.11	.06	−.26*	−.13	−.15
4	Perceived competence in written argumentation				1	.42**	−.04	.17	−.07	.04	−.11
5	Perceived instruction in argumentation					1	−.09	.23	.05	.05	−.08
6	Prior beliefs						1	−.12	.13	.15	.21
7	Essay – argument quality							1	.26*	−.05	−.03
8	Essay length								1	.19	.14
9	Recall – inferential clauses									1	.23
10	Recall – total clauses										1

*p < .05, **p < .01

The relationship between thinking dispositions and multiple‐texts comprehension

The research question was investigated with a path analysis approach. AOT, NFC, perceived competence (a factor composed of perceived topic knowledge, perceived exposure to argumentative essay instructions, and perceived competence in argumentative writing), and prior topic beliefs were included as independent variables. We estimated their effects on argumentative essay quality and the number of valid inferences recalled. We also estimated the association between the number of valid inferences and the total number of clauses to control the potential confounding effect of a higher number of inferences due to longer recall protocols. The essay length was included as a mediator. The estimated path model had a good fit [χ2 = 27.71, p = .37; RMSEA = .03; CFI = .98]. Although the three perceived competence measures were intercorrelated, one may object that they differ in how they influence the outcomes. Indeed, one may consider himself or herself very competent in argumentative writing but not so knowledgeable in the specific topic assigned and vice versa. Thus, we compared this model with a less parsimonious one in which we estimated the associations between the three measures of perceived competence on the argumentative essay quality and length (see Figure 2).

Figure 2

Expected (less parsimonious) model (NFC = need for cognition; AOT = actively open‐minded thinking).

This model had a good fit too [χ2 = 23.49, p = .37; RMSEA = .03; CFI = .90]; however, it was less parsimonious (22 degrees of freedom instead of 26 of the former model) and had a lower CFI.2 Thus, we chose the former model, which include a perceived competence factor on which the three perceived competence measures saturate, to investigate the research question (see Table 3; Figure 3). NFC was positively associated with the essay length [β = .33, p < .01], and AOT was positively associated with the number of valid inferences recalled [β = .28, p < .05]. The essay length was positively associated with the argumentative essay quality [β = .31, p < .05]. Neither AOT nor NFC were directly associated with argumentative quality; however, NFC was indirectly associated with argumentative quality via the essay length [β = .10, SE = .06, p = .05, 95% CI = 0.01; 0.20]. Perceived competence was positively associated with argumentative quality [β = .26, p < .05] but not with the essay length. Argumentative quality and valid inferences recalled were not significantly associated. Overall, effect sizes were small‐to‐moderate.

Table 3

Path analysis model results (n = 73)

Path	β	SE	95% CI
Perceived competence BY
Perceived knowledge of topic	.74***	.10	0.58; 0.90
Perceived competence in written argumentation	.86***	.10	0.71; 1.02
Perceived instruction in argumentation	.54***	.11	0.37; 0.72
Prior Beliefs ON AOT	.19	.12	−0.01; 0.39
Essay – argument quality ON
Perceived competence	.26*	.14	0.04; 0.48
Prior beliefs	−.16	.12	−0.36; 0.04
Essay length	.31*	.13	0.09; 0.53
AOT	.14	.13	−0.07; 0.35
NFC	−.10	.13	−0.31; 0.11
Essay length ON
Perceived competence	−.17	.15	−0.41; 0.07
AOT	.16	.13	−0.05; 0.37
NFC	.33**	.11	0.14; 0.51
Recall – inferential clauses ON
Essay – argument quality	−.08	.12	−0.28; 0.12
AOT	.28*	.12	0.09; 0.47
NFC	.11	.12	−0.09; 0.31
Recall – total clauses	.20	.12	0.01; 0.39

*p < .05, **p < .001, and ***p < .001.

Figure 3

Path analysis model with significant standardized path coefficients (NFC = need for cognition; AOT = actively open‐minded thinking; *p < .05).

Discussion

The present study investigated the association between thinking dispositions (AOT and NFC) and multiple‐texts comprehension (as measured through an argumentative essay and a recall task). Poor performances in the argumentative essay and the recall task confirm the difficulty of university‐level students to integrate opposing stances on a controversial topic (Bråten et al., 2011; Mateos et al., 2018). The two most relevant results were that (1) NFC was directly associated with the essay length and indirectly associated with argumentative essay quality, and (2) AOT was directly associated with valid inferences recalled.

The essay length appears to play a relevant role in students' multiple‐texts comprehension behaviour (MacArthur et al., 2019). Since elaboration is defined as enriching text information with additional associations, it may be assumed that elaboration takes more space in the written text than would reproducing facts (Lonka & Mikkonen, 1989). In an argumentative essay task, essay length may be assumed as an indirect measure of generalized effort (Eisenberger et al., 1982) or knowledge of the topic and awareness of its complexity (De La Paz et al., 2017). The low performances in argumentative essay quality and the number of valid inferences and the lack of association between the two tasks seem to suggest that levels of elaboration were low. Thus, on a speculative level, we suggest that higher levels of NFC were associated with higher levels of argumentative essay quality through either or both a higher task awareness and a higher effort.

Thinking dispositions were also associated with the number of valid inferences recalled. AOT may support depth of comprehension through epistemic self‐regulation, inducing a disposition to go beyond our own beliefs, and consider new scenarios. The ‘active’ component may also induce a higher agency to the reader, making him/or her more active in the processing of texts. Whereas NFC seems associated with a higher effort and/or engagement, AOT seems associated with a higher elaboration of both perspectives (Stenhouse et al., 2018). Unexpectedly, AOT was not associated with the level of intertextual integration. Past studies focused on improving argumentative writing suggested that interventions are more effective if both dispositions and skills are targeted (Hefter et al., 2018). Thus, AOT may be not sufficient to improve the quality of students' argumentative essay if not coupled with argumentative skills (i.e., declarative and procedural knowledge about argumentation).

In this study, the argumentative essay quality was not significantly associated with valid inferences recalled. Whereas this result may seem unexpected, past studies have found that students' argumentative thinking (in the form of evaluation of arguments) was not associated with recall, probably because the information was not deeply elaborated (Diakidoy et al., 2015, 2017). This lack of association is problematic because students seem to struggle in creating an integrated situation model of the texts, despite the level of integration between arguments discussed in the texts. Integration may be achieved with different levels of elaboration, through refutations (the lowest level of elaboration) to minimal or overall weighting and synthesizing (the highest levels of elaboration; Mateos et al., 2018; Nussbaum, 2008; Nussbaum & Edwards, 2011). The highest level of elaboration in the argumentative essay task (i.e., more frequent use of weighting and synthesizing integration strategies) may lead to deeper comprehension in the recall task.

Perceived competence was significantly associated with argumentative quality, in line with prior studies that found an association between task‐specific self‐efficacy beliefs and multiple‐texts comprehension (Bråten et al., 2013). Indeed, how well students believe they can do on achievement tasks directly influences performance, effort, and choices of which tasks to pursue (Wigfield & Eccles, 2000). Finally, prior topic beliefs were not significantly associated with any of the variables included in the study. This may depend on the choice of the topic (i.e., evaluation of teachers). Students were moderately pro‐evaluation, but a scientific topic may trigger stronger biases (see also Diakidoy et al., 2017).

Limitations and directions for future research

When interpreting the findings of the current study, some limitations should be taken into account. The first limitation depends on the sample size. While the sample size could be considerate adequate for the path analysis model tested in this study (Sideridis, Simos, Papanicolaou, & Fletcher, 2014; Tabachnick & Fidell, 2007; Wolf, Harrington, Clark, & Miller, 2013), it may have had not enough statistical power to detect further indirect effects (Wolf et al., 2013). The second limitation depends on the topic chosen for texts assigned to students. Socio‐scientific topics may have triggered more biased beliefs, which could have influenced the involvement of thinking dispositions. Future studies should replicate the present research design by varying the controversial topic to assign to students. Moreover, readers' performances change when texts are reading real documents rather than print‐out versions of multiple texts (Salmerón, Gil, & Bråten, 2018). Thus, future studies should explore the relationship between thinking dispositions and multiple‐documents (rather than multiple‐texts) comprehension in real documents. Finally, future studies should focus more on the processes and strategies adopted by students when they integrate and how they are associated with the construction of an integrated situation model as represented in recall tasks.

Theoretical and educational implications

Despite the limitations, the present study contributes to the literature on multiple‐texts comprehension by emphasizing the role of thinking dispositions. A strength of the present research design is represented by the implementation of both a proximal and a distal measure of text processing, allowing to investigate whether thinking dispositions play a different role in different multiple‐texts comprehension outcomes. NFC was indirectly involved in both multiple‐texts comprehension tasks, and AOT was involved in the depth of comprehension in the recall task, suggesting a proximal role for NFC and a more distal role for AOT. Moreover, this study confirms that the essay length may be a variable of theoretical interest, as also shown by its association with thinking dispositions.

On an educational level, the results of this study point out the necessity of successful instructions, which help students to learn how to integrate (Barzilai et al., 2018), especially in writing (Van Ockenburg, Van Weijen, & Rijlaarsdam, 2019). Furthermore, the present study contributes to our reflection on the difficulties of undergraduate students in multiple‐texts comprehension, which may partially depend on the disconnection between argumentative writing and recall. Students may only superficially elaborate arguments presented in the text to argue in support of one position only. This study also contributes to renew educational practitioners' attention on essay length as a sign of task awareness or effort. Although it is possible to write short essays of good argumentative quality, thresholds for essay lengths could be identified and be used as a ‘rule of thumb’ by teachers, along with strategy instruction on how to compose a good argument. Finally, the study emphasizes the contribution of two thinking dispositions, a malleable component of the human mind, that is hypothesized to be improved through admonition or instructions (Stanovich & West, 1997), with expected direct and indirect positive effects on students' performances in multiple‐texts comprehension.

Conflicts of interest

All authors declare no conflict of interest.

Author contribution

Christian Tarchi, Ph.D. (Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Resources; Validation; Writing – original draft; Writing – review & editing) Ruth Villalón (Conceptualization; Data curation; Formal analysis; Methodology; Writing – original draft; Writing – review & editing).