Reflection: A Socratic approach.

Reflection is a fuzzy concept. In this article we reveal the paradoxes involved in studying the nature of reflection. Whereas some scholars emphasize its discursive nature, we go further and underline its resemblance to the self-biased dialogue Socrates had with the slave in Plato's Meno. The individual and internal nature of the reflection process creates difficulty for studying it validly and reliably. We focus on methodological issues and use Hans Linschoten's view of coupled systems to identify, analyze, and interpret empirical research on reflection. We argue that researchers and research participants can take on roles in several possible system couplings. Depending on who controls the manipulation of the stimulus, who controls the measuring instrument, who interprets the measurement and the response, different types of research questions can be answered. We conclude that reflection may be validly studied by combining different couplings of experimenter, manipulation, stimulus, participant, measurement, and response.

Ever since Plato, the philosophically good life has been propagated as a life that is permeated by reflection, that is, a life that aims at wisdom, at finding truth about our existence, and achieving a moral balance in our actions (Plato, 1997). “Man’s place in creation” was an accepted object for reflection. St. Augustine’s confessions (circa AD 400), for instance, demonstrated a possession with reflections on the relationship of the individual person to God and their conscience (Pusey, 1999). Ages later it became “Man’s place in Nature” (Huxley, 1906), and Edmund Husserl tried to find the foundation of science and logic in unbiased reflection (e.g., Husserl, 1900–1901, 1936).

Reflection stems from the Latin verb reflectere, meaning “to bend” or “to turn back on the self” (Harper, 2001–2017). From an etymological origin a variety of meanings were derived, including whether and how persons can know or relate to themselves (Wiley, 1994). Reflection can be considered “a highly personal, cognitive process… in which a person takes an experience from the outside world, brings it inside the mind, turns it over, makes connections to other experiences, and filters it through personal biases” (Dewey, 1910/1933, p. 9). Definitions of reflection often depict a cognitive activity of questioning (e.g., Boud, Keogh, & Walker, 1985; Daudelin, 1996) alleged to have explorative, developmental, and interactive merits (e.g., Boud et al., 1985; Boyd & Fales, 1983; Brookfield, 1988). The role of self is prevalent in making sense of one’s own experiences (e.g., Boud et al., 1985; Boyd & Fales, 1983; Daudelin, 1996; Seibert & Daudelin, 1999).

These definitions illustrate that reflection is considered to be a highly individual and internal process resembling a dialogue with oneself, and mainly inaccessible to others such as researchers. Nevertheless, we found reflection to be a tool of many trades. The process of reflection as well as its effects have been studied by scholars in philosophy, pedagogy, management, and organizational psychology. Many studies either explicitly or implicitly refer to reflection as an inner dialogue between the self as a subject and an object (or the object’s processes). Due to the nature of reflection, however, it is hard to avoid the temptation to use one’s reflective capacities to study reflection, and, like in Socrates’ dialogue with the slave, to interrogate oneself, thus becoming one’s own slave. The subject of reflection scores high as a psychologist’s dilemma: to subject the self to science or science to the self (Leary, 1990). On the one hand, no mental activity asks for more validity and reliability for both science and self; yet, this can easily be confused (Halligan & Oakley, 2015; Krause, 2005).

What is needed for reflection to be reliable and valid is an anchor point in a “reality” that is accessible to others for critical discussion. Ultimately, both researchers and reflecting individuals want reliable and valid conclusions about the perceptions, feelings, arguments, conclusions, and thoughts of their inner selves, often referring to these as “honesty” or “integrity.” In what follows, we suggest a useful approach to analyzing and categorizing the conceptual and methodological problems of studying reflection that was inspired by Hans Linschoten’s ideas of coupled systems[1] (1964). Linschoten considered the relationship between a psychological researcher and their experimental “participant” as one in which two persons have roles in several possible system couplings. Application of his coupled systems approach to a methodologically diverse set of reflection studies reveals possible improvements to studying reflection that we will describe in the last part of this paper.[2]

Selected studies on reflection

Reflection is a popular research topic across various scientific disciplines. Within these disciplines, methods like experiments, surveys, and qualitative studies have been used to uncover it. With the intention to provide a cross-sectional image of reflection studies, we selected a methodologically diverse sample of the literature on reflection. Table 1 provides an overview of 18 articles that served as a basis for our discussion.

Table 1.

Selected studies on reflection and inclusion criteria.

Author	Journal	Method	Reflection stimulus		Intervention technique
Anseel et al. (2009)	Organizational Behavior and Human Decision Processes	Experiment	Feedback	F	Coached reflection	C
Bruno et al. (2011)	European Journal of Psychology of Education	Qualitative study (content analysis)	Become a trainer “for a day”	P	Writing a personal journal	J
D’Argembeau et al. (2005)	NeuroImage	PET, (f)MRI	Focus mental activity on specific topics	S	Reflective tasks	TP
Davis & Klaes (2003)	Journal of Economic Methodology	Essay/literature review	There is more than one possible strategy to respond to the various dimensions of reflexivity.	E	Considered as phenomenon rather than intervention technique	TP
Grant et al. (2002)	Social Behavior and Personality	Survey	Journal or diary keeping	S	Questionnaire	SR
Gürtner et al. (2007)	Organizational Behavior and Human Decision Processes	Experiment	Training by means of guided reflection: team-based military air-surveillance task	P	Reflexivity intervention	TP
Jordan (2010)	Management Learning	Qualitative study (ethnography, narrative)	Explicit and indirect reference to diversity and the necessity of asking questionsMonthly rotationInteractive on-the-job trainingAlternation of peripheral and full participation (learning-by-doing)Case-based teachingEmphasis on reconstructing abstracted rules	P	Question routinized waysInteractive practices	C
Kember & Leung (2000)	Assessment & Evaluation in Higher Education	Survey	Class evaluation	F	Questionnaire	SR
Larrivee (2008)	Reflective practice	Survey	Emotionally supportive learning climateMediation processes	F	PromptsJournalingNon-judgmental questioning	J
Livengood et al. (2010)	Philosophical Psychology	Experiment	Personality, philosophical training	P	Cognitive reflection test	SR
Lyke (2009)	Personality and Individual Differences	Experiment	Psychotherapy	E	Questionnaire	SR
Mann et al. (2009)	Advances in Health Sciences Education	Essay/literature review	Awareness of a need or disruption in usual practiceAnticipatory phaseAppropriate supervisionNovel or challenging situationsIntellectually and emotionally supportive environment	P F	Observational methodsAnalytical methods	SRC
Masui & De Corte (2005)	British Journal of Educational Psychology	Experiment	Training	P	Questionnaire	SR
Procee (2006)	Educational Theory	Essay/literature review	Reflection partnersExerciseEmpathy	P	Intelligence (Kant’s Verstand)Judgment (Kant’s Urteilskraft)	TP
Rogers (2001)	Innovative Higher Education	Essay/literature review	CoachingUncertainty	E	JournalingRole modelingUse of questionsCritical incidents	J
Schippers et al. (2007)	Applied Psychology: An international review	Survey	Team work	E	Questionnaire	SR
Trapnell & Campbell (1999)	Journal of Personality and Social Psychology	Survey	Self-focus	S	Questionnaire	SR
Van Woerkom & Croon (2008)	Personnel Review	Survey	Experience	P	Questionnaire	SR

Note: Column 4: F = Feedback, P = Practicing, S = Self-focus, E = Evaluation Column 6: C = Coaching, J = Journaling, TP = Task performance, SR = Self-report.

A comparison of the articles suggests that reflection has frequently been studied in the scientific domains of education, management, psychology, and philosophy. Psychology and management studies often focused on reflection effects (e.g., Anseel, Lievens, & Schollaert, 2009; Masui & De Corte, 2005) and the development and validation of reflection measures (e.g., Schippers, Den Hartog, & Koopman, 2007; Trapnell & Campbell, 1999). Contributions in the education literature pursued the identification of reflective practices (e.g., Bruno, Galuppo, & Gilardi, 2011; Rogers, 2001). Some articles addressed reflection’s neurological location (e.g., D’Argembeau et al., 2005), personality features (e.g., Livengood, Sytsma, Feltz, Scheines, & Machery, 2010), and logical pitfalls or paradoxes (e.g., Davis & Klaes, 2003).

In general, reflection is considered to be a cognitive process to which individuals have to provide some effort (e.g., Bruno et al., 2011; D’Argembeau et al., 2005; Grant, Franklin, & Langford, 2002; Gürtner, Tschan, Semmer, & Nägele, 2007). In some studies, the affective process has been addressed as well, in order to stress the emotional factors at stake (e.g., Jordan, 2010; Kember & Leung, 2000; Mann, Gordon, & MacLeod, 2009; Rogers, 2001; Trapnell & Campbell, 1999). Apart from the interest in individual, cognitive, and affective processes, some articles discuss collective and reflective processes (e.g., Schippers et al., 2007; Van Woerkom & Croon, 2008).

Conceptual comparison reveals three different approaches toward reflection. The first approach involves reflection in the presence of self-awareness. This approach emphasizes the investigation of personal knowledge structures by means of introspection (e.g., Mann et al., 2009; Procee, 2006). The second approach considers reflection in terms of self-reference. Self-reference pays attention to the self–other relationship and stresses the evaluative rather than critical nature of reflection (e.g., Gürtner et al., 2007; Van Woerkom & Croon, 2008). The third approach is referred to as self-inquiry. Self-inquiry includes “epistemic” interest in the self by means of questioning assumptions one previously has taken for granted (e.g., Livengood et al., 2010; Trapnell & Campbell, 1999).

The different concepts of reflection have been classified according to timing (e.g., Jordan, 2010; Rogers, 2001; Schön, 1983, 1987), cognitive effort (e.g., Larrivee, 2008; Schippers et al., 2007), and level of reflection (e.g., Davis & Klaes, 2003; Kember & Leung, 2000).

Stimuli and intervention techniques give some indication of what makes people reflect. Practicing or learning-by-doing is considered an important stimulus for reflection (e.g., Jordan, 2010; Masui & De Corte, 2005). As such, reflection is used as a learning strategy, from which one can benefit the most when thinking back afterwards or even instantly about the effect or efficiency of the practices at hand. Feedback provided by relevant others (e.g., Mann et al., 2009) as well as behavioral evaluation (Schippers et al., 2007) are frequently mentioned as stimuli for reflection (see Table 1). Intervention techniques for reflection include self-reports (e.g., Kember & Leung, 2000), evaluating task performance (e.g., Gürtner et al., 2007), coaching (e.g., Jordan, 2010), and journaling (e.g., Bruno et al., 2011).

The main purpose scholars have attributed to reflection is the assessment of cognitive functioning (e.g., D’Argembeau et al., 2005; Trapnell & Campbell, 1999). Here, reflection is used to determine a person’s critical thinking skills. Other purposes are sense-making (e.g., Bruno et al., 2011; Jordan, 2010) and redirection or self-regulation skills (e.g., Anseel et al., 2009; Procee, 2006). Livengood et al. (2010) have emphasized the importance of reflection as a merit or legitimization of one’s (cognitive) behavior. Other studies have suggested that reflection enhances performance. In their article, Mann et al. (2009) mentioned mixed effects of reflection on performance. This supports the negative and mixed effects from reflection on well-being (e.g., Lyke, 2009; Trapnell & Campbell, 1999). With regard to understanding, again some studies demonstrate mixed or positive effects (e.g., Davis & Klaes, 2003; Grant et al., 2002; Jordan, 2010; Livengood et al., 2010; Rogers, 2001).

The above suggests that a person reflects to qualify their own cognitive functioning in terms of task performance and critical understanding and to manage their own behavior. When a person’s thinking ends in vicious circles, reflection hampers well-being rather than improving performance. All studies, however, seem to be caught in a chicken-and-egg situation: They suggest that reflection has positive results for cognitive functioning and for performance, but depend on the very same person’s reports about their cognitive functioning, their performance, or the reception of a stimulus—if not incentive—to reflect.

Problems studying reflection

Recent discussions seem to concentrate more on the methodological nature of the challenge than on the deeper problem residing in research on reflection. Experimental tasks that aim to investigate reflection vary enormously within and across domains. They vary from demonstrations of internalized learning of study skills (Masui & De Corte, 2005), to written individual and group discussions (Gürtner et al., 2007), elaboration with and without feedback (Anseel et al., 2009), solving math problems (Livengood et al., 2010), and reflecting on the self, others, or social issues during positron emission tomography (PET) scans (D’Argembeau et al., 2005). With the exception of the neuropsychological experiment of D’Argembeau et al. (2005), they show little relation with self-reflection as studied through surveys. Only some of the studies have tried to distinguish levels of self-reflection. Moreover, many confounding factors surface, such as new ways of processing information (e.g., by email), using multiple-choice items (which seems a distinct type of activity from reflection), or learning to learn, which was not related to reflection as such. There was only limited transfer (or the measurement thereof) to other domains and limited long-term knowledge of applying reflection in resembling domains. Furthermore, authors found hardly any distinction between individual and group processes. In other words, a unifying paradigm has not yet been found, and if it is found it will not be easily related to the qualitative variations involved in reflection. The reporting of procedures, participants, and results does not always conform to what unequivocal conclusions require.

As alarming as these methodological problems may be, however, they may obscure a deeper problem in reflection research. This has to do with the nature of reflection and the temptation to use one’s reflection capacities to study reflection and, like Socrates reflected in interrogating oneself, thus becoming one’s own Socratic slave. Simultaneously addressing who reflects on what and what is represented reveals a paradox in reflection, however, resembling the infamous reflective paradox involving self-contradicting premises (Ashmore, 1989; Bartlett & Suber, 1987). Combining the premises “Lying means not telling the truth,” and “I now do not tell the truth,” results in the paradox “I am now lying (about my telling the truth).” Since the act of combining both premises and the act of “lying” refer to the same person (the self), the truthfulness of the conclusion is uncertain. In other words, the subject and the object of lying become identical. In reflection issues, the same seems to be the case: the actor and the observer of reflection are identical.

Turning ourselves into objects overlooks the fact that this act of objectification is subjective in itself (Cunliffe & Jun, 2002). Or, in Hegelian terms, the subject has a permanent blind spot. Hegel illustrated this by a reflecting pole that can only see the other pole (the “reflectee”) and not itself (Wiley, 1994, p. 79). Or, as Kant’s picture metaphor illustrates, “whenever the knower tries to know the knower it thereby turns the knower into the known. The knower knows the known, but not the knower” (as cited in Organ, 1987, p. 115). Apparently, we can only discriminate these two elements in an analytical sense and not in a physical or statistical sense when confronted with locus issues.

Linschoten’s system couplings

The problem may be illustrated with a view we borrowed from Dutch psychologist Hans Linschoten. Linschoten (1964) observed that psychologists seem to believe the position that all psychological knowledge about human beings is self-implicating. That is, psychologists’ scientific knowledge is also referring to themselves, and therefore self-committing and self-biased. In other words, psychologists should apply whatever they know about human beings to psychologists as well. Consequently, the biases and prejudices that psychologists observe in human beings have to apply to psychologists as well (i.e., “Idols of the psychologist”[3]).

Linschoten considered the relationship between a psychological researcher (E) and their experimental “subject” (O; now “participant”) as one in which two persons have roles in several possible “system couplings.” In an experiment (Figure 1(a)), a psychologist with the role of experimenter E has control over stimulus situation (x) and measuring instrument (m), whereas the observed participant (O) “controls” the response (r) to E’s manipulation (x). Figure 1(b), (c), (d), and (e) reflect other couplings in which psychologists gain knowledge about the validity of the instrument (Figure 1(b)), the reliability of the observer by comparing observations with those of others (Figure 1(c)), and properties or traits of the participant, as in using questionnaires (Figure 1(d)). In system couplings in Figure 1(c) and (d), E does not manipulate the stimulus to know its effect, but studies reactions to a “given” stimulus in order to get more insight into the properties of the measuring instrument or into the characteristics of the participants. Below, we further discuss the different couplings of systems involved in research, and give some examples.

Figure 1.

Five ways that an experimenter, his instruments, and his participant can be coupled. See the text for an explanation.

Examples of coupled systems in reflection studies

Reflection has been studied by means of experiments, surveys, and qualitative studies. Researchers determine what to measure, how to measure, and how to interpret measurement outcomes; participants are subject to measurement and sometimes are (non)deliberately able to influence measurement outcomes. Clear-cut as it seems, however, there is still some leeway. Earlier, we mentioned paradoxes and blind spots that are typical for reflection. The paradoxical nature of reflection is not restricted to the object of study, but often also involves measurement. For instance, how autonomous is the “self” that initiates reflection? Is reflective behavior probed by researchers distributing reflective tasks similar to “spontaneous” reflective behavior that is self-reported by participants? Whose reflective behavior is at stake: that of the research participants, the researchers, or both? For example, on the one hand participants involved in an empirical study on their reflective behavior are subject to paradoxes and blind spots such as biased memories. Researchers, on the other hand, do not only have to identify participants’ blind spots, but also have to deal with their own paradoxes and blind spots. Linschoten’s coupled systems are informative to analyze these issues in empirical studies.

Below we give some examples of reflection studies that can be characterized by the way systems are coupled. We do not pretend to cover all the studies and mixed methods published. To illustrate how in reflection research systems are coupled differently, we only discuss some striking examples.

Coupled system #1: The experiment

In experiments, the researcher (E) has control over the stimulus (x) and the measurement phase (m). After stimulation, the participant (O) responds (r). The focus in this coupling is on the effect of x (independent variable) in terms of r (dependent variable), and m is used to establish the effect.

For instance, in an experiment by Masui and De Corte (2005) young participants had to formulate study recommendations for themselves and for peers, and establish relations between their personal study behavior and good and weaker study results in two previously followed domain-specific courses. The experimental treatment involved training sessions in—as well as practice and transfer tasks on—self-regulated learning. These sessions and tasks included the self in relation to social relations and the self in relation to consequences of study behavior (“the future profession”). In this coupled system (#1), the experimenter had control over the stimulus (a set of instructions) and the measurement (attribution and reflective behavior measurements, as well as academic achievements); the participant had control over the response, that is, his reflective behavior. According to Linschoten’s systems criteria, Anseel et al. (2009) conducted an experiment investigating the effect of reflection on performance. They provoked experimental “reflection” in feedback or no-feedback task conditions. The authors (E) defined four task conditions (x) in which participants (O) had to select an answer to different work emails and: (a) write down examples of their own strong and weak task performance based on performance feedback (reflection/feedback task); (b) do so without such feedback (reflection/no feedback task); (c) not write down anything but receive feedback; or (d) write down and not receive feedback. Next, they had to do similar email-response test tasks (r), and performances (m) between the four different feedback/reflection conditions were analyzed. Feedback with reflection generated the highest test performance.

Coupled system #2: Calibration

In calibration studies, the main aim is to establish the validity of the instrument (Figure 1(b)). In other words, the psychologist is interested in the precise way the measuring instrument relates to responses by participants. Studies use the measured strength of the stimulus in relation to the kind and/or strength of the response. The focus in this coupling is on the validity or standardization of m, using the effect of x (independent variable) on r (dependent variable) in the case where m is used to establish the effects of r.

One way to validate findings in reflection research is to triangulate methods, such as interview, observation, and document analysis (see Denzin, 1978). In an extensive study, Carson and Fisher (2006) “raised the bar” by indicating how the reflective quality of student writings can be established by using identification procedures of values, beliefs, assumptions; changes in these values, beliefs, and assumptions; their making connections to political, social, and cultural values; and changing their habits. Scott empirically tested the usefulness of portfolios as learning tools. The research question in this “calibration study” was straightforward: “Is student portfolio use positively associated with reflection?” (Scott, 2009, p. 62). In other words, can we use portfolios to establish serious reflection? Portfolios were offered to students as a stimulus to reflect on learning and performance in an MBA course. The effect on reflection performance was established by measuring the level of reflection exhibited in an exam paper. Apparently, that was not the only aim of her study. The portfolio stimulus used in the study was also validated by a qualitative evaluation of the students’ excerpts from their portfolios to establish whether the stimulus actually elicited reflection.

Coupled system #3: Self-calibration

According to Linschoten, self-calibration takes place when the focus of the researcher is on improving the interpretation of the measurement instrument. In a more straightforward measurement situation the focus is on preventing “reading errors” of instruments, the correct interpretation of test results, and so forth. The experimenter does not control the stimulus, but is interested in the question of whether he or she rightly interprets the measured response.

Christie and Menhuir (1997) studied (E) what a “reflective practitioner” means and how the process of reflection can be operationalized and evaluated in the context of the continuing professional development (CPD) of those who work in the field of early education. This is an example of how teachers (O) can be supported to evoke (x) more reflection techniques in their pupils (r), and at the same time it gives an example of how they can “self-calibrate” their method of establishing progress (m) in their students. Another example is Jordan’s study (2010) who, as a participant observer (E) observed and interviewed (m) novice nurses (O) of an anesthesiology department regarding how they would become reflective practitioners, addressing different dimensions of reflective attitudes, and how to interpret the “measurement” of them. Jordan investigated how organizational practices (x) fostered reflection-in-action in a high reliability context (r), by categorizing her field observations of seven novice nurse anesthetists (verbal and nonverbal communication), narratives, interviews, or document analyses into different theoretical dimensions that structured the thick descriptions (an ethnographic approach).

Coupled system #4: Measurement

There are many examples of studies in which there is negligible or no stimulation by the researcher. The researcher, however, controls the measuring instrument, as in a survey or questionnaire, and establishes any response of a participant that can be interpreted as a reflective practice. Another way to ascertain an external effect more independently is to use modern neuropsychological technology during reported reflection in order to obtain more objective measurements of reflective activity. For example, Northoff et al. (2006) and Saxe, Moran, Scholz, and Gabrieli (2006) used neurological measurements. In their neuropsychological experiment, D’Argembeau et al. (E) used three topics to evoke (x) reflection: reflection on one’s personality, reflection on someone else’s personality, and reflection on social issues. To identify the dependent variable (r)

immediately after each scan, subjects were asked to verbally report the thoughts, images, and/or memories they had had while they were reflecting on the topic. Then they rated several aspects of the mental activity they had experienced during the scan, including amount of thoughts (i.e., total amount of thoughts experienced, whatever their content); relative amount of thoughts about the self; thoughts about other persons; memories; physical sensations and combined those ratings with measurements of brain metabolism patterns through PET scans. (D’Argembeau et al., 2005, p. 620)

They inferred that self-reflection differs from reflection on others or society and from a resting state, although during rest some similar patterns were visible, both in the metabolism scans and in the self-ratings. Apparently, self-reflection plays an important role during rest.

Coupled system #5: Spurious coupling

For our study, spurious coupling is a most relevant system coupling. Linschoten referred to number 5 as “spurious” because, as he called it, “science appears as power” (Linschoten, 1964, p. 157). No reliable or valid knowledge is collected (Linschoten, 1964, p. 157). The psychologist or their client wants a certain result of the study, and stimulates and measures just long enough to attain the wanted result. His examples seem to refer to applying power that disguises as scientific knowledge, as you often see in advertisements of dubious products, using the phrase “scientifically proven.” For instance, we forbid our client to use the word polar bear and we observe that the client does not use the word polar bear, or worse: we report that the client does not think about polar bears. In our study, reflection could border on spuriousness if we use reflections to “prove” how—or that—reflection has the effect we wished for. For instance, in a study on distributed shared sense-making, Ladewski, Krajcik, and Palincsar (2007) theorized that

inquiry and reflection are mutually constitutive processes that play a key role in human flexible shared sense-making, enabling sense-making systems (individual and collective) to “open” to consider multiple possible options and then to “close” to the “best fit” option for a particular situation based on criteria such as reproducibility, coherence, and fruitfulness. (Ladewski et al., 2007, pp. 45–46)

They tested their theoretical frame by documenting reflections and prediction or what they called “sense-making activities” of students and not collecting other performance data or data from comparison classrooms. In a post-graduate course for pharmacists, Black and Plowright (2010) exposed their students to a systematic reflective learning strategy and a written reflective portfolio. Next, they asked focus groups and individuals in an interview to reflect upon reflective learning, resulting in a multidimensional model of reflection. They argued that reflective learning is more complicated than usually described. In both cases it seemed that a model of reflection that the investigators already must have had in mind was used to reflect on the model of reflection that was under construction. It is a Socratic situation.

Table 2 classifies types of studies based on Linschoten’s coupled systems, including an extended meretricious coupling (discussed below).

Table 2.

Classification of empirical studies on reflection, based on Linschoten’s coupled systems, with one added by the present authors.

Coupled system	Researcher influences the experimental intervention X	Participant influences response R	Measurement’s function
1. The experiment	Yes	Yes	Researcher tests effect of intervention
2. Calibration	Yes	Yes	Researcher tests validity of measurement
3. Self-calibration	No	Yes	Researcher tests validity of his/her own interpretation
4. Measurement	No	No	Researcher measures an external, non-controlled for effect
5. Spurious coupling	Yes	No	Researcher intervenes and measures to obtain a wanted result
Addition:6. Meretricious coupling	No	Yes	Researcher = participant

An additional system: The meretricious coupling

In addition to the original set of five couplings, our analysis suggests a sixth one. In this situation, the participant’s (or subject’s) role (O) cannot be distinguished from the role of the psychologist (E), and the measurement (m) and response (r) have faded into the mind of the experimentalist/object as well. The psychologist as his/her own object, as it were, administers his/her own stimulus (x) and has the response “in his/her head,” and designed the measurement as well. As a name, we propose “the meretricious coupling.” We suspect that the meretriciously coupled system relates to spurious coupling. The reflecting individual actually plays two if not three roles: to be their own psychologist, to be their own object, and to be their own judge of successful reflecting. In spurious coupling, two parties are involved, the researcher and the client who wants to get the result of the study that he or she demanded; in meretricious coupling, both parties are one and the same person. Figure 2 demonstrates the spurious interdependency of meretricious coupling.

Figure 2.

Meretricious coupling #6 (inspired by Linschoten, 1964).

Conceptually, this seems to be the basic problem, if not paradox, with reflection as a technique for thinking about our own behavior, thinking, learning, and its improvement (in the eyes of which beholder?) In the Cartesian situation, I can observe another’s behavior (but not their inner experience), and they can observe my behavior (but not my inner experience). This resolves the question of whether we deal with a knower who is open to the external world (although it is internally represented), or with a self-contained knower, closed to the world, for whom all things that seem real are the mere product of imagination and construction: Descartes versus Vico, Popper versus Wittgenstein, Munz versus Rorty (Munz, 1984, 1985; Rorty, 1980). Regardless of whatever insight we gain about ourselves, we can only be our own judges of the validity of insight. In terms of objectivity, there is no way that, for insight and the test of its validity, traces of the knower can be removed or ignored.

Therefore, we cannot reasonably be sure that we are honest and reliable instruments for the assessment of the truthfulness of our reflections. Notwithstanding its personal worth and explorative value, all personal diaries, log books, and other journaling, blogging, and vlogging initiatives fit this meretricious coupling, at least whenever authors use them as arguments for the validity and reliability of reflection.

We consider it defendable that any attempt to design experiments that go deeper than looking at the effect of alleged reflection tasks will result in serious problems as soon as reflection is seen as a mental process in which words are used as definite indicators of inner thought processes. Experiments that report effects of what is only roughly indicated as “reflection” may demonstrate that instructing to reflect can help. They do not demonstrate that reflection works, let alone how it works. Many seem to claim to be effect studies but fail to be so and are, at best, calibration or self-calibration studies.

A way to approach this problem is to look at the opportunities offered by triangulation, for instance with neuropsychological data. It is not our aim to study brain processes during mental activities to find the exact spot where reflection is located. Neuropsychological evidence, however, can help to validate whether or not an alleged mental process corresponds with a related brain process (Wegner, 2002). We discuss this approach in the next section.

Implications

From the above we can conclude that the concept of reflection can have many different meanings and interpretations. Is reflection considered thinking? Is it conscious thinking? Or thinking about consciousness? Consciousness of what (DaSilveira, DeSouza, & Gomes, 2015)?[4] Does reflection occur intentionally and, if it does, to what extent should we consider it truthful if it concerns only conscious and intentional thinking? How do we manage self-bias? Is the scope of reflection restricted to actions rather than to thinking?

The objects of reflection, and especially the self as object, are often subject to discussion. Reflection is doubted as a reliable source of representing reality, including one’s own place in reality. Most of our cognitive processes are inaccessible to consciousness. Since there are, for now, no means of observing the contents of personal thought independently of thought, the self-reflecting person is trapped in a reflexive paradox. This includes the psychologist and results in considering reflection to be a state of mind where attitudes and intentions are invented as explanations of one’s behavior during events or phases one happened to participate in. It results in a self-constructed story.

Research program on reflection

To overcome the problems of the self-implicated nature of reliable and valid psychological knowledge about reflection, and the “meretriciousness” of reflection, we will now briefly suggest additional ways to investigate the true nature of reflection. The actual issues are twofold: (a) does the participant really reflect on his/her focal problem; and (b) what processes are involved in reflection?

Based on Linschoten’s coupled systems, we suggest an “ideal” research design, consisting of multiple research waves. The first wave involves a basic research design that refers to coupled system 1. Next, subsequent waves serve to separately validate Linschoten’s coupling elements X, O, r, m, and E. As such, coupled system 1 is more or less “quintangulated” to arrive at an ideal design for a research program in which each separate element is controlled for (Figure 3). For instance, the reflective task or prompt (x) is compared with a self-reflective task—asking the participant to suggest a relevant question for him/her to start reflecting on (cf. content validity). Participants’ (O) consequent and unadulterated responses to the reflective and self-reflective tasks are checked by means of combining intentional verbal responses with unintentional physical responses (cf. criterion validity). To validate participants’ responses (r) in themselves, comparisons with “relevant others,” for example, friends and relatives (cf. 360-degree feedback) as well as between cultures could be helpful. With regard to measurement (m), combination of multiple methods such as functional magnetic resonance imaging ((f)MRI; e.g., what are the active brain parts), eye-tracking (e.g., selective and first attention), and verbal responses (e.g., self-reports, diaries) could be informative. Finally, the activities of the researcher or experimenter (E) are validated by testing his/her interpretations and first focus (e.g., eye-tracking of reading (f)MRI results) in relation to other relevant researchers (cf. inter-encoder or inter-rater reliability).

Figure 3.

“Ideal” research design in which coupled system 1 is “quintangulated.”

For time, access, and budget restrictions, a research design containing five waves may be utopian in itself. Each wave, however, addresses subsequent questions[5] that are necessary to answer in order to disentangle reflection. Even a triple wave design using (f)MRI or PET scans can help to avoid some of the pitfalls and biases that impair many studies on reflection. For instance, being involved in a reflective task, research participants could be shown a photograph expressing a universal emotion. Next, participants’ immediate responses could be traced by measuring a physiological response ((f)MRI), signalling first attention (eye-tracking), and asking for a comment (interpretation). This would help researchers detect becoming aware of the activities involved in reasoning about a certain human emotion. To investigate a self-reflective task would involve another wave in which participants are invited to provide feedback on their own behavior. For instance, participants could be asked to describe and explain their emotions when seeing the photograph. Additionally, they could be encouraged to suggest some reflective questions themselves, which in their mind would help to make sense of the photograph.

Although it is not our aim to find the location of the “organs” of reflection (if they exist at all), we hypothesize that there are processes involved in reflection that differ from, for example, “mere” thought or problem-solving. To investigate, two elements must be observed. One concerns the nature of the “experimental task,” the other the probable nature of reflection as a Socratic dialogue.

Traditionally, the approach has been to ask the participant to reflect on a certain problem and report the results. Concerning the experimental task, we suggest that the approach taken long ago by Shepard (1982) and Shepard and Cooper (1986) to present participants with a binary imagery problem and ask for the right interpretation, can act as an example. Shepard and Cooper presented their participants briefly with a 3D block structure, and then briefly with a rotated version of it, or with a rotated and mirrored version. Participants were to answer the question whether or not the second structure was the mirrored version of the first structure. The experimenter’s measurement was whether the participant was correct, which led to an unequivocal conclusion: does he/she use mental imagery effectively or not? Transposing this to the reflection problem would lead to experiments in which the participants are asked to use reflection to “solve” a problem with two possible answers, one of which the experimenter knows is right. Similarly, regarding the task used by Livengood et al. (2010), who presented participants with mathematical problems from the Cognitive Reflectivity Test developed by Frederick (2005), it could be argued that this involves not so much reflection, but mere problem-solving or thought. To deal with that, we would like to suggest several comparisons. One comparison is between (f)MRI scans of participants’ activities while solving mathematical problems from the Cognitive Reflectivity Test and (f)MRI scans of participants’ activities answering the following question: “think of what questions the researcher should ask you to establish what you are considering when reflecting on a subject of your own choice.” The latter question is an open question. It prevents the criticism, however, that participants do not reflect but only solve the problem of what to say when the experimenter asks them to reflect about a specified subject. It is our impression that a question like this invites true reflection. Additionally, we suspect that the comparison of the (f)MRI scans of both tasks indicates different brain processes that might be involved in reflection other than mere thought or problem-solving, thus giving the opportunity to check on “real” reflective activity.

Though very difficult to realize, another more complicated way to study the phenomenon allows answers to some deeper questions. In the meretricious coupling situation, the supposed subsystems of experimenter and participant are not independent but coupled systems, as they are in an experimental situation. Actually, experimenter and participant are one and the same person. In philosophical terms, this is the reflection situation in optima forma. With an open attitude, and with every self-inspired question to ask oneself, an honest and fair answer is possible. Psychologically speaking, however, there are many reasons why this will not result in honest and fair answers. The temptations of self-protecting and self-indulging answers, let alone self-implicating biases and self-betrayal, are unavoidable. Following the interesting suggestions made by D’Argembeau et al. (2005), our suggestion is to investigate the involvement of the ventromedial prefrontal cortex (vMPFC) in meretricious coupling. That is, the experimenter/participant should be followed during the whole process (or all phases of the process) involved in reflection. The reason for this is that, on the one hand, it appears necessary to gain insight into the reflection process as a whole from self-posed questions to self-given answers and, on the other hand, to control for self-serving biases. Technical and practical problems provided, we think these are what conceptually seem to be necessary steps. It gives room for the phenomenologically or philosophically suggested typical features of reflection, while it constrains the experimental situation sufficiently to provide reliable and valid answers.

Caveats and conclusions

Reflection on reflection

One caveat that can be made to our suggestions is that reflection has been given many meanings. As one reviewer of our original manuscript adequately remarked, reflection and self-reflection should be distinguished. Of course, the reviewer is right. We think, however, there is no fundamental difference. Both involve the self as the locus or subject of a reflection as well as the object or focus of the reflection. One could reflect on, say, the sentence, “Even the president of the United States sometimes must have to stand naked” (Dylan, 1965, 7th stanza, Lines 5–6), but this would only be called reflection if it involves the self’s own inner experiences with that sentence. If not, it would be called, perhaps, critical thinking or artistic interpretation. It naturally implies that reflection always involves a reflecting person’s thoughts and other experiences. “The self” cannot be the focus of a reflection other than, either, the word self (in which case it is a thought of a person about a word), or of the self (in which case it is a thought of a person about a person who happens to have the same identity). The first case is trivial, the second probably impossible.

Blind spot (reflective paradox) of psychology

Another caveat concerns the reflective paradox of psychology, or what could be called its blind spot. Indeed, as William James identified as “the great snare of the psychologist” (James, 1890, p. 196), it is easy for psychologists to confuse what their participant thinks with what the psychologist thinks. When I see a hole somewhere under a tree, it is easier for the psychologist to think that their participant also sees a rat hole, if the psychologist knows about rats and their holes (as many of them seem to do). It is the blind spot of the psychologist to not realize that his or her participant might just see a rabbit hole. Again this includes what can happen in reflection research: is it reflection in the eye of the beholder or in the eye of the psychologist?

Socrates’ slave and Socrates’ self

Having found multiple problems at several levels in studies of reflection, we conclude that a more thorough understanding of the conceptual problems involved in reflection is needed. Our analysis suggests that reflection could still be seen as a substitute dialogue with another person. The psychology of reflection might be inspired by methods used for a century or more in penal law. Asking a person to self-reflect with the possibility that a third person might “fact check” all assertions might help to unravel the methodological and conceptual knots that psychology is in. It would be of key value for attaining positive results.

Another inspiration might be found in the way the use of mental imagery is tested by presenting participants with two-choice questions about 3D structures (Metzler & Shepard, 1982; Shepard, 1982). It would resemble the Socratic method, though not so much as a rhetorical device, but as a forced choice designed to check true reflectivity.

A different approach will come from neuropsychology. We do not suggest that reduction of neurological events gives definite answers. Nevertheless, neurological evidence can help to control for claims about the relationship between behavior and internal intentions, reflected or otherwise.

In general, claims that reflection helps to improve action by learning to better respond to complex situations could benefit from triangulation, if not quadrangulation or quintangulation, of methods that are insufficient on their own. Reflection as an inner argument needs anchor points to reality. As in law, stories of what happened (who did what, why did he do it, is it punishable behavior?) need anchor points for a judge or jury to decide on prosecution.

Overall, we suppose that reflection is truly Socratic. Superficially, it resembles a dialogue and could be called discursive, as suggested elsewhere (Larrain & Haye, 2012). On a deeper level, we suggest that reflection resembles the actual content of what Socrates presented to the docile slave. Socrates gave him rhetorical questions, that is, answers phrased as questions to which the obedient slave could only answer with a “yes” or, depending on the question, with a “no.” The slight difference with the “real” Socrates is that our inner Socrates cannot avoid any demand characteristics (Orne, 1962), and that our inner slave has a self-interest in the answer.