First Validation of the Referential Process Post-Session Scale - Therapist Version (RPPS-T).

OBJECTIVE: Bucci's multiple code theory maintains that for a significant change the patient-therapist relationship should foster a referential process shaping in three alternating phases: arousal of emotion schemas, symbolizing/narrating emotional experiences, and reflecting/reorganizing the emotional meanings. Until now to monitor these phases clinicians and researchers have used several referential process computerized linguistic measures, which however need the sessions verbatim transcription. In order to have a less time-consuming method we developed and tested a therapist self-report questionnaire measuring the referential process phases.
METHOD: We asked eight psychotherapists in a first study and nine psychotherapists in a second study to complete the Referential Process Post-session Scale - Therapist version (RPPS-T) just after the end of their sessions. In a third study we transcribed 29 sessions conducted by three psychotherapists to calculate the correlations between the RPPS-T scores and the computerized linguistic measures of the referential process calculated on the session transcripts.
RESULTS: In the first study we collected 105 evaluations regarding 29 patients and an exploratory factor analysis revealed a four-factor pattern consistent with the hypothesized constructs. The analysis reduced the initial pool of 42 items to 12. In the second study 130 sessions with 25 different patients have been evaluated on the shortened version of the RPPS-T and a confirmatory factor analysis found that the four-factor model satisfactorily fitted the new data as well. In the third study we found that the factors of RPPS-T regarding the symbolizing phase correlated with the corresponding computerized linguistic measures calculated on the session transcripts.
CONCLUSIONS: The RPPS-T received a first validation as a concurrent measure of the referential process, especially for the symbolizing phase, and could be considered a useful instrument for research and supervision.

Introduction

The key role of referential process in psychotherapy

The Multiple Code Theory (MCT; Bucci, 1997a, 2002, 2013a, in press-a; Bucci & Maskit, 2007) is a general theory of emotional information processing and psychotherapeutic change, consistent with the most recent advances in cognitive psychology, psychoanalysis and affective neuroscience. According to this theory human beings have manifold codes relating to the various sensory channels and different organizing systems that combine information from these channels at different levels of complexity and abstraction. The continuous and indistinct flow of emotional and sensorial experience is initially elaborated by the subsymbolic system in which multiple information is processed simultaneously in a global and analogical modality, along continuous dimensions. From the continuous flow of subsymbolic experience the nonverbal symbolic system draws discrete units like images and representations. The verbal symbolic system translates into words, even if only partially, these images and representations. In humans the information processing is characterized as what Bucci (Bucci, 2013a; Bucci at al., 2016) calls the Referential Process (RP), namely the ability to translate the continuous and indistinct flow of emotional and sensorial experience into images and words, thus making it mentally manageable for oneself and communicable to others. This process, however, is partial because it is not possible to completely translate all emotional and sensory experience into words. Thus, a certain amount of disconnection always remains between what people feel and what they can think and communicate of their experience (Bucci, 2003, 2013b). Moreover, when a person experiences painful events and emotions some processes aimed at distancing and removing painful sensations are automatically activated and in this case the disconnection recursively increases and may become more permanent. Hence, in the interpersonal history of people, especially those with a significant emotional and interpersonal rigidity, some emotion schemas develop and are pervasively activated at the subsymbolic level. This may be seen in interactions within the therapy context, where the person is not able to associate experiences to clear images and representations and to describe them in words. The greater the disconnection, the more the person feels emotional arousal without being able to mentally manipulate and communicate the experience being aroused.

In interpersonal relationships (Bucci, 2005, 2013b; Bucci & Maskit, 2007), and in particular in the therapeutic one, communication takes place both at the subsymbolic and symbolic levels. The disconnected emotion schemas of patients–and of therapists as well–can be communicated and jointly experienced through the interconnection of the two speakers’ subsymbolic systems. Thanks to the activation of the RP of both partners in the therapeutic relationship, the disconnected emotional experiences can find new paths of connection through images and words. A productive therapeutic session, in which this process is activated (Bucci, 1997a; Bucci & Maskit, 2007; Bucci, et al., 2016; Kingsley, 2010; Kris, 1956), would involve movement through arousal, symbolizing and reorganizing phases. In the arousal phase, the patient experiences some sensory and bodily elements of emotion schemas and appear to be struggling with painful feelings and ideas associated with them; in the symbolizing phase patient begins to find images and words to describe what they are feeling, and narratives of event, memories, dreams or fantasies may take the stage; and finally patients, also thanks to the therapist's interventions, pass to a phase of reorganization and reflection that expands the meanings attributed to the experience, creating the potential for new connections and changes in emotional organization to be made.

There is a vast amount of recent empirical findings coming from the fields of cognitive science and affective neurosciences that are convergent with this model of emotional information processing and interpersonal communication. For example, research in the areas of mirror systems (Rizzolati, Fadiga, Fogassi, & Gallese, 2002), enactive perception (Kinsbourne & Jordan, 2009), and embodied communication (Jordan, 2009) provides evidence for the bodily and interpersonal foundation of the subsymbolic communication of the emotions in the interpersonal contexts. Moreover, the recent intersubjective perspective in psychoanalysis has contributed to theories of changes (e.g. Bromberg, 2006; Greenberg, 2011; Hoffman et al., 2016; Stern, 2004) that are consistent with the RP phases described.

Assessing referential process

Over the last decades Bucci and colleagues (Bucci & Maskit, 2006; Maskit, in press; Zhou et al., in press; Mariani et al. 2013; Negri et al. 2019) have developed and validated several computerized linguistic measures that allow researchers to visualize, monitor, and measure these phases of the RP, and therefore to measure the emotional elaboration process put in place in the therapeutic conversation. In particular the two measures of the RP that have been most well developed at this time are the Referential Activity (RA) and the Reflection and Reorganization Function (RRF).

RA has been defined as the degree to which the speaker or writer is able to translate their emotional, visceral and relational experience into words, so as to evoke corresponding experiences in the listener or reader (Bucci, 1984; Bucci & Freedman, 1978; Bucci et al., 2016; Bucci, in press-a, in press-b). The higher the concreteness, clearness, specificity and imagery of discourse, the more the words work to translate the subsymbolic experience in an immediate way for listeners. This measure is central in representing the symbolizing phase of RP.

RRF can be defined as the degree to which the speaker is trying to recognize and understand the emotional significance of an event or set of events in their own or someone else’s life, or in a dream or fantasy (Zhou et al., 2018; Negri et al., 2018). It is not abstract and logical reasoning but active searching for subjective meanings of a memory in which the speaker is emotionally engaged. This measure is useful to represent the reorganizing phase of RP.

Several other complementary measures–some of which are detailed below in the measures section of Study 3–have also been developed by the authors to capture other aspects of RP. For example, the dictionary of the disfluencies is particularly useful for highlighting the arousal phase in which the ideas and images are in a nascent state and the speaker often hesitates in finding the words and clarity necessary to start a narrative to represent their emotional activation; the dictionaries of affects, reflection and sensory-somatic sensations composed of words relating respectively to emotions, abstract logical functions and bodily sensorial perceptions help to identify the speech referring to bodily and emotional experience in an abstract way. In addition, a set of derived measures have been developed that indicate the intensity and pervasiveness of RA and RRF or compute covariations between the different measures. An important value of all these measures is that they refer primarily to aspects of language style and so are applicable across a wide range of contents. In addition, neither the speaker nor listener are likely to be aware of these aspects of language, and so they provide a window to experience that is out of awareness.

The use of computerized linguistic measures has shown many advantages including the analysis of large collections of texts and conversations through an automated procedure that is rapid and not biased by the subjective interpretation of researchers. This methodology has made it possible to analyze the RP in a wide range of contexts and on different materials from clinical and non-clinical subjects. By way of example, several studies investigated RP through single cases or psychotherapists’ notes analyses (Bucci et al. 2012; Di Nardo et al. 2005; Hoffman et al., 2013; Lo Verde et al., 2012; Mariani & De Coro, 2013; Rocco, 2018; Rocco et al. 2013; Rodriguez Quiroga De Pereira et al., 2018), other authors investigated the RP measures in relation to specific pathologies (Ben-Meir et al., 2009; Di Trani et al. 2018; Fertuck et al., 2004; Gorska & Soroko, 2017; Jepson & Bucci, 1999; Mariani et al., 2020) or in relation to other relevant clinical constructs such as the therapeutic alliance and reflexivity and the psychotherapy outcome (Christian et al. 2010; Esposito et al., 2019; Negri, & Mariani, 2005; Negri et al. 2019; Negri et al. 2020; Renzi et al., 2020; Sammons et al., 1998; Vegas et al., 2015).

Aim and hypotheses

As the literature shows, till now to monitor the development of the three phases of RP psychotherapists and researchers have relied on their own clinical sensitivity and computerized measures of RP which require the transcription of sessions. These measures require however that the conversations to be analyzed are recorded and transcribed. Many therapists, particularly psychodynamic therapists in private practice do not choose to record their treatments, and the transcribing process that is required for computer analysis is time consuming and costly. Even for studies applying the computerized measures to treatment notes, technical issues may also arise. We therefore wondered if it might be possible to develop a self-report measure of the phases of the RP that was sufficiently reliable but less time consuming than studies involving the computerized measures. Self-report assessment tools are generally less time consuming and easier to use than studies applying objective measures, but give results that are more simplified and often biased by the responders’ attitude; this is particularly the case for studies involving patient self-report. However, previous studies (Blagov et al., 2012; Westen & Weinberger, 2004) have suggested that clinicians tend to make more reliable evaluations if their observations and inferences are systematized through psychometrically grounded instruments. Moreover, the therapists’ self-reflective abilities with respect to what was experienced in the session could lead to more reliable evaluations than those of the patients. If sufficient reliability could be obtained, a self-report measure of the RP could contribute to evaluation of the process, and also provide a highly useful and informed point of view. The therapist in fact who is involved in the psychotherapy process as much as the patient, could be asked through a self-report questionnaire to evaluate the RP immediately at the end of the session, when the memory of the session is still active and alive, and working through processes are operating.

In this paper, we present three studies in which we sought to examine whether the RP could be measured by a questionnaire completed by psychotherapists at the end of the sessions.

The aim of Study 1 was to develop a first version of the Referential Process Post-session Scale – Therapist version (RPPS-T) comprising three dimensions covering the arousal, symbolizing and reorganizing phases of RP (Bucci, 1997a, 2019; Bucci et al., 2016). We expected by analyzing a dataset of scores given through RPPS-T to find three dimensions corresponding to the three phases of RP.

Study 2 was aimed to test if the factor pattern found in Study 1 was confirmed on a different set of scale administrations and to measure the reliability of the scale. We expected that the factors–if confirmed–were intercorrelated since all RP phases are likely to occur at some point in a session showing a productive RP (Bucci, 1997a; Bucci & Maskit, 2007).

In the Study 3 we sought to study the convergent validity by exploring the correlations between the RPPS-T scores and the computerized measures of RP when both are applied to the same sessions. In particular in accordance with the literature (Kingsley, 2010; Mariani et al., 2013; Maskit, in press) we expected that the factors corresponding to the arousal, symbolizing and reflection/reorganization phases to show positive correlation with the disfluency, referential activity, and reorganization/reflection scores respectively as derived from the computerized linguistic analysis of the session transcripts.

Study 1

Method

Measures

An initial extended Italian version of the Referential Process Post-session Scale (RPPS) was developed creating a set of items concerning the three phases of RP. The aim was to develop a self-report instrument to be repeatedly applied by the psychotherapist at the end of each session to monitor the RP along the treatment course. The dimensions covered by the first set of 42 items were based on the definition of RP phases (Bucci, 1997a; Bucci, 2000; Bucci et al., 2004; Negri et al., 2018; Zhou et al., in press). Six items were generated for the arousal phase. The symbolizing phase were represented by six items for each subscale of the referential activity measure; these are concreteness, specificity, clarity, imagery. Twelves items were created for the reflection/reorganization phase.

The instructions for psychotherapist were: “RPPS should normally be completed as soon as the session is over or at most by the end of the day of the session. Please read carefully the following statements, some concerning the in-session joint (patient and therapist) experience and some concerning your sensations at the end of the session as therapist. For each statement mark a number between 1 and 7 to say how much you disagree or agree with the statement. Use the following rating scale, with 7 if you strongly agree, and 1 if you strongly disagree. The midpoint, if you are neutral or undecided, is 4”.

Participants and procedure

We recruited eight psychotherapists through a specific call addressed to professional and scientific associations of which the first author is a member. They varied with respect to their clinical experience (ranging from two to 15 years of psychotherapy practice), all had a psychodynamic clinical orientation, and had acquired general knowledge of the MCT but had no attended courses or training specifically focused on the MCT and RP measures.

We requested that they select among their patients one or more who were at least 18 years old, who had had no psychotic disorder or psychotic symptoms for at least the previous six months, and whom the therapist had seen for a minimum of six sessions. Once one or more patients meeting the criteria had been selected, we asked psychotherapists to complete the 42-item extended version of RPPS-T at the end of their sessions for at least one time. Psychotherapists received no remuneration. All participants provided a written informed consent and were seeing the selected patients in a private setting.

Results of exploratory factor analysis

From the eight psychotherapists who evaluated 29 patients we collected 105 evaluations. An exploratory factor analysis was conducted to explore the 42-item RPPS-T dimensions. We examined solutions for four and five factors applying the principal axis factoring extraction method and we preferred the four factors solution, which explained 64.0% of the variance, because it met the criteria of the initial eigenvalues higher than unity (Kaiser, 1960) and the "leveling off" of eigenvalues on the scree plot was identifiable after four factors (Cattell, 1966). We then compared both Varimax and Promax rotations and decided to use the Promax rotation because it showed an expected correlation between the four factors and because it provided the best-defined factor structure. We retained only the items that contributed to a simple factor structure with primary factor loadings of .4 or above, and with no cross-loading of .3 or above (Osborne, 2014). Among the set of items that met these criteria we retained only the three items with the highest primary loadings for each factor in order to make the questionnaire as short as possible and with items equally distributed among all four factors. For the final stage, another exploratory factor analysis of the remaining 12 items was conducted, resulting in the four factors explaining 63.3% of the variance. All items in this analysis had primary loadings over .48 and cross-loadings below .27. The pattern matrix for this final solution is presented in . The four factors explained 17.6%, 16.8%, 14.5%, and 14.5% of the variance respectively and had a satisfactory internal consistency (α = .822, .989, .781, .803 respectively; Bland & Altman, 1997; Cronbach, 1951).

Considering the contents of the items comprising in the factors extracted (), the authors agreed upon the following names: (1) Concreteness and Imagery (Con/Im), (2) Reflection and Reorganization (Ref/Reor), (3) Specificity (Spe), (4) Vividness of the Session Memories (Mem). Of the three RP phases only two are clearly represented by the dimensions emerged: the reflection and reorganization phase by factor (2) and the symbolizing phase by factors (1) and (3). Conversely, a dimension relating to arousal phase did not emerge; however, the factor (4) relating to the vividness of the session memories could be considered indirectly connected with emotional arousal in a general sense: the more emotional activation in the session, the more likely it is that the experience of that session will be remembered vividly. We note that this is different from the qualities associated with the arousal phase in the context of the referential process. We will return to this issue in the discussion of results for the paper as a whole.

Study 2

In this study the 12-item version or RPPS-T resulting from the exploratory factor analysis described in Study 1 () was administered with the same instructions as used for the original version (see Method section of Study 1).

Nine psychotherapists different from those involved in Study 1 were asked to complete the 12-item RPPS-T through a specific call addressed to professional and scientific associations of which the first author is a member. The psychotherapists recruited had different amounts of experience (ranging from four to 22 years of psychotherapy practice) and different clinical orientation (six psychodynamic, one systemic, two cognitivist). Two of them were expert of the MCT and RP measures, and they are two authors of the present article. Also in this study we requested that they selected among their patients one or more who were at least 18 years old, who had had no psychotic disorder or psychotic symptoms for at least the previous six months, and whom the therapist had seen for a minimum of six sessions. Once one or more patients meeting the criteria had been selected, we asked psychotherapists to complete the 12-item version of RPPS-T at the end of their sessions for at least one time. Psychotherapists received no remuneration. All participants provided a written informed consent and were seeing the selected patients in a private setting.

Results of confirmatory factor analysis

The nine participants completed the 12-item RPPS-T at the end of 130 sessions with 25 different patients. On these 130 administrations we conducted a confirmatory factor analysis to test if the factor pattern found in Study 1 was confirmed also in the new dataset. Stevens (2002) and Hu and Bentler’s (1995) guidelines for various fit indices were used to determine whether the expected model fits the data. The chi-square test statistic was used, but considering its sensitivity to sample size, other fit indices were evaluated: (a) the comparative fit index (CFI ≥ 0.90 for a good fit); (b) the root mean square error of approximation (RMSEA ≤ 0.10 for an acceptable fit); and (c) the standardized root mean square residual (SRMR ≤ 0.08 for an adequate fit).

The four-factor model with 12 items found in Study 1 satisfactorily fitted the new data, χ2 (48) = 105.395, p < .001; CFI = 0.940; RMSEA = 0.097, 90% CI [0.072, 0.122]; SRMR = 0.049. This factor pattern was thus considered as the final model for the RPPS-T. Figure 1 shows the significant standardized regression coefficients of each items on the respective factor and the significant correlation estimates between factors. As expected, the four dimensions positively correlated, and this result was confirmed also by the correlation matrix between factors computed on the raw scores (). The internal consistency of all factors was satisfactory (Bland & Altman, 1997; Cronbach, 1951), Con/Im (α = .791; M = 4.83; DS = 1.24), Ref/Reor (α = .797; M = 5.49; DS = 1.21), Spe (α = .876; M = 5.79; DS = 1.19), Mem (α = .778; M = 5.72; DS = 1.12).

Figure 1.

Standardized Factor Loadings and Correlation Estimates of RPPS-T Final Model

Study 3

Bucci and colleagues (Bucci & Maskit, 2006; Bucci & McKay, 2004) have developed and validated several computerized linguistic measures of RP which have also been developed in various languages, including Italian. The Italian Discourse Attribute Analysis Program (IDAAP; Maskit, in press) compares a text in Italian with word lists or dictionaries, including weighted dictionaries (Bonfanti et al., 2008; Di Trani et al., 2018; Mariani et al., 2013; Negri et al., 2018). The output is a list of counts and average weights indicating the proportion of these words in the text. For this study we apply the Italian version of the following dictionaries or linguistic measures of the RP.

Weighted Referential Activity Dictionary (WRAD).

For each text, WRAD provides the average of the weights that the words assume in terms of referential activity; these are the weights related to their degree of concreteness, specificity, clarity and imagery. As noted, referential activity is defined as the degree to which the speaker or writer is able to translate their emotional, visceral and relational experience into words, so as to evoke corresponding experiences in the listener or reader (Bucci et al. 2004; Bucci, 2000). It is a measure of emotional involvement based on the connection between emotional experience and words. WRAD scores range from 0 (lowest) to 1 (highest RA) with .5 as the neutral value. High levels of WRAD are associated with the symbolizing phase of RP.

Weighted Reflection and Reorganization (WRRL).

WRRL is a dictionary of words associated to the Refection and Reorganization function. This can be defined as the degree to which the speaker is trying to recognize and understand the emotional significance of an event or set of events in their own or someone else’s life, or in a dream or fantasy; it is not about abstract reflection but rather a person’s reasoning related to an experience that has been vividly experienced (Negri et al., 2018; Zhou et al., in press). WRRL is an index of personal elaboration of emotional experiences and its scores range from 0 (lowest) to 1 (highest RRF) with .5 as the neutral value. High levels of WRRL are seen as indicators of the reflection and reorganization phase of RP.

Mean High Weighted Referential Activity Dictionary (MHWRAD).

MHWRAD is calculated on the basis of WRAD scores and is a measure of high intensity of emotional engagement (Mariani et al., 2013) emerging from the speech. It indicates how high the WRAD is when it is above the mean. It is obtained by looking only at the words with WRAD scores lying above the neutral value and then computing, for only those words, the average level of the IWRAD scores. This can be understood as a measure of intensity of immersion in the symbolizing function of the RP.

Mean High Weighted Reflection and Reorganization List (MHWRRL).

MHWRRL is calculated on WRRL scores and is a measure of the high intensity of the reflection and reorganization function as emerges from a person’s speech. It indicates how high the WRRL is when it is above the mean. It is obtained by looking only at the words with IWRRL scores lying above the neutral value and then computing, for these words only, the average amount of the WRRL scores. It is best understood as a measure of intensity of immersion in the reflection and reorganizing function.

Disfluency Dictionary (DfD).

DfD is a small set of words as well as repetitions of words, incomplete words and filled pauses that people tend to use when struggling to communicate. This index corresponds to the proportion of disfluency words present in the speech. High scores typically characterize the arousal phase in which the emotion schemas are activating.

Reflection Dictionary (RefD).

RefD is a list of refection-related words; it provides the proportion of words in the text referring to cognitive or logical functions, and to communication processes that imply the use of cognitive functions. RefD is a measure of the abstract reflection present in a speech.

Sensory Somatic Dictionary (SenSD).

SenSD is a list of words related to the body and bodily activities, and to sensory processes and/or descriptions of symptoms. The number of SensD words in the speech sample is a measure of the arousal of bodily subsymbolic aspects of emotion schemas.

We asked the participants involved in Study 2 if they were available to record the sessions that they would evaluate through RPPS-T in order to assess the convergence between RPPS and scores derived from the applying of the RP computerized linguistic measure on the session transcripts. Three psychotherapists, with a psychotherapeutic experience of over 10 years and a psychodynamic orientation, agreed to propose to their patients to record some of their sessions; two of the psychotherapists are authors of the present article. A total number of 18 sessions with four different patients were recorded and transcribed verbatim. All sessions were also evaluated using RPPS-T by the psychotherapists just after the session. Psychotherapists received no remuneration. All psychotherapists and patients provided a written informed consent and were seeing the selected patients in private clinical practice.

Results

The correlations of the four factors of the RPPS-T as evaluated by the therapists with the linguistic measures of RP computed on transcripts of the same sessions are shown in . The direction of the associations is partially consistent with expectations.

In Studies 1 and 2 no scale was clearly associated with the arousal phase. As noted earlier, this function of the RP is the least clearly defined by Bucci and her colleagues in terms of its connection to language (Bucci et al., 2016; Bucci, in press-b, in press-c; Maskit, in press). We note here that if we consider the Mem factor as indirectly connected to the emotional activation in the session we see that it correlated negatively with SenS (r = -.503, p = .002) and positively with MHWRAD (r = .444, p = .007). The vividness of the memories about the session therefore increased consistently with the intensity of the in-session referential activity and decreased as the words connected to sensory and bodily sensations increased.

As regards the symbolizing phase, the factor analysis confirmed the presence of two factors–Con/Im and Spe–corresponding to three of the four subscales (concreteness, specificity, clarity, and Imagery) composing the RA index representing this phase. The Con/Im factor correlated negatively with SenS (r = -.605, p < .001), WRRL (r = -.492, p = .002) and MHWRRL (r = -.483, p = .003), and positively with DfD (r = .336, p = .045); the Spe factor correlated negatively with SenS (r = -.402, p = .015), WRRL (r = -.365, p = .029) and MHWRRL (r = -.363, p = .030) and positively with MHWRAD (r = .391, p = .019). If we consider the Con/Im and Spe factors together, computing their average scores, they correlated negatively with SenS (r = -.559, p < .001), WRRL (r = -.473, p = .004) and MHWRRL (r = -.466, p = .004), and positively with MHWRAD (r = .361, p = .031). There was thus a substantial convergence between the RPPS-T scales measuring the concreteness, imagery and specificity and the intra-session referential activity intensity. In contrast, a substantial divergence was found between these RPPS-T factors and the intrasession reflection and reorganization function; furthermore, also in this case there was a negative correlation between these RPPS-T factors and the use of words referring to sensory and bodily sensations.

The Ref/Reor factor that should measure the reflection and reorganization phase correlated negatively with SenS (r = -.547, p < .001) and positively with MHWRAD (r = .383, p = .021). Thus, for this factor was not found the expected correlation with the dictionaries shaping the correspondent phase–WRRL, MHWRRL and Ref–whereas Ref/Reorg factor resulted correlated with the main index of referential activity–WRAD. It appears that the clinical raters using the RPPS-T may have been emphasizing the aspect of the reflection and reorganization function that is based on the presence of an emotional experience being thought about and discussed rather than abstract and intellectual reflection. Lastly, also in this case a negative correlation was found between this factor and the use of words referring to sensory and bodily sensations.

,

Discussion

The new instrument we developed to measure the RP phases characterizing the change process in psychotherapy (Bucci, 1997a, 2000, 2013; Bucci et al., 2016) has received a first validation from the three studies we conducted, and has also raised interesting questions to be explored further. Of the three RP phases we intended to capture with the RPPS-T, two are clearly represented by the dimensions confirmed by the factor analyses: the reflection and reorganization phase by one factor and the symbolizing phase by two factors. A fourth factor–the Vividness of Session Memories–can be indirectly associated to the arousal phase if we consider that an intense emotional activation during a session likely lead to clear and vivid therapist's memories of what happened in that session. These findings add new and interesting information to the conceptualization of the arousal function of the RP; the connection of this function to language also remains the least well understood in the work by Bucci and her colleagues.

The RPPS-T four factors we found–Concreteness and Imagery, Reflection and Reorganization, Specificity, Vividness of Session Memories–showed a well-defined factor matrix and satisfactory internal consistency values. Correlations among these factors are significant and coherent with the MCT theory-based expectations. The results of the confirmatory factor analysis showed the validity of the four-factor structure of the scale, even if the chi-square test statistic was significant and the RMSEA was at the extreme level of the range for acceptability. In fact, chi square test statistic is highly sensitive to sample size and other fit indices must be evaluated. Similarly, there is little empirical support for the use of universal cutoff values for RMSEA and their confidence intervals to determine adequate model fit since its values depends on model specifications, degrees of freedom, and sample size (Chen et. al., 2008). Marsh et al. (2004) recommended to consider different fit indices together to evaluate overall goodness of model-data fit. Thus, from a psychometric point of view RPPS-T can be considered a valid and reliable instrument.

In the Study 3 we tested also the convergent validity of the new scale examining the correlations between the RPPS-T scores and the computerized linguistic measures of the RP as applied to the same psychotherapy sessions. The correlation matrix indicated a substantial convergence between the constructs measured by the two instruments.

The RPPS-T factors relating to the symbolizing phase–Concreteness and Imagery, and Specificity–and their average combination, correlated positively with the intensity measure of RA and negatively with the intensity measure of RRF. These results are consistent with the theory-based expectations as the moments in which patient reorganizes the emotional experience and reflects on it are different from and usually follows the symbolizing phase which is largely characterized by an immersion into the narrative process. These results provide confirmation that these two RPPS-T scales capture and measure the symbolizing function and also discriminate it from sessions dominated by the reorganization and reflection function. The fact that these two scales did not correlate with the RA general measure–WRAD–but only with the measure of the RA intensity–MHWRAD–indicates they detect the degree to which patients experience an intense emotional involvement in narrating their emotional experience rather than the general tendency to symbolizing in a session. Thus these two scales can be considered a measure of the RA intensity rather than a general measure of the RA. These results are also consistent with ongoing work by Bucci and her colleagues relating the measures to clinical evaluations. Some of this work was reported by Bucci in recent presentations in Milan and Rome as well as by Bucci (2017). The positive correlation between the in-session disfluency, considered an emotional activation measure, and these two RPPS-T symbolizing scales provides some confirmation that patients tend to be highly emotionally engaged during the narration of their experiences.

Related to this point, we note that the RPPS-T Reflection and Reorganization factor showed a significant correlation with the RA intensity measure (MHWRAD), but did not show a correlation with the linguistic measures–WRRL and MHWRRL–which are directly and manifestly related to the reflection and reorganization construct. This result support the suggestion that the Reflection and Reorganization scale as rated by clinicians focuses on the personal and active emotional engagement necessary to a have a meaningful reorganization phase. It may be more difficult for the therapist to grasp with the RPPS-T the more reflective aspect of the conversation held during the session as part of the function assessed by the RPPS-T. In addition, we should note that the computerized linguistic measures of the RP and the RPPS-T are instruments used to measure the same constructs but at two different times, from two different points of views, and on two different objects, respectively the in-session and the post-session elaboration processes. Like a dream could be different from a memory of a dream it is plausible that we have some differences in the measurement of the complex phenomenon of the RP. In this sense also the discovery of a different and unexpected concept, as represented in the Vividness of Session Memories factor, indicates that the memory processes and their quality are crucial in the measurement of the RP when one is outside the immediacy of the interpersonal encounter.

Another interesting result about the relationship between RPPS-T scales and the RP linguistic computerized measures is that all four RPPS-T scales highly and negatively correlated with the proportion of words relating to bodily and perceptual sensations (SenSD). Mariani et al. (2000) found that high SenSD scores characterized severely depressed patients’ autobiographical narratives and in general evidenced speeches with an operational, non-metaphorical or non-emotionally connoted storytelling style. We can therefore suppose that high SensD scores indicate a tendency to manage the emotional suffering in a psychosomatic way rather than in a symbolized way. The words relating to bodily and perceptual sensations, thus, are likely to be frequent when patients make greater reference to symptoms and their bodily correlates. This could explain why SenSD index is inversely associated with the RPPS-T scales measuring the RA and RRF functions and with the vividness of session memories. If patients in fact talk about their symptoms this at least initially takes away space from the narration of life experiences and prevent a more intense elaboration of subjective meanings (Bucci, 1997a, 1997b). On the other hand, as Bucci (1997b) has noted, description of symptoms may function in a useful way as a path to symbolic functioning, particularly for patients for whom the connection of experience to language may be more difficult. The different possible implications of high levels of SensSD language remain to be explored.

In general, our findings indicate that RPPS-T has the potential to be a valuable tool for the future research in psychotherapy, filling a gap that has existed between standard self-report measures and computerized assessment of transcribed session material. Research on patient’s and therapist’s emotion elaboration in fact represents one of a more promising field of investigation that could highlight the main elements associated to clinical outcomes (Lane et al., 2015; Rottenberg & Gross, 2007). To achieve this goal multi-perspective and multi-method studies are needed in order to integrate outcome with process research, qualitative with quantitative analysis, third-parties with subjective measures of the process, and patient and therapist’s contributions to the psychotherapeutic change process (Lepper & Riding, 2006; McLeod, 2013; Peluso & Freund, 2018). The RPPS-T is a tool that can measure the outcome of psychotherapy by detecting the sessions and treatments where a good referential process is evident and where it is not; it can also be considered a process measure as it indicates which aspects of the session process are not working; and it represents an integration of objective analysis of the RP, such as that provided by computerized linguistic measures, with the therapist’s subjective ratings of the process.

Limitation and future research directions

The three studies we conducted have some limitations that could be overcome in the future research on this instrument. First, the therapists and patients involved are few. In order to investigate the construct validity of the RPPS-T it is necessary to have participation of more therapists with different clinical orientations with application of the measures of a larger sample of patients and psychotherapy sessions. A second limitation was that the sample in Study 2 and 3 included two of the authors of the study who were aware of the hypotheses. Further, in Study 3 knowledge on the part of therapists and patients that the sessions would be recorded could have partly changed the course of the sessions. Lastly, specific and personal emotional dynamics of the therapists involved in the three studies or even the normal memory distortion processes can make difficult for the therapist to reliably assess the RP of the session when it is over. The possibilities and limitations of measuring the RP by the actors of the session themselves remain to be examined. The method of this project, which incorporates the assessment by objective measures is a step towards investigation of these possibilities and limitations to be continued with further studies.

In future work, the relation of RPPS-T to other validated instruments measuring important and theoretically related constructs such as emotion regulation, reflective functioning, therapeutic alliance, and countertransference could be examined. This would allow to further investigation of the reliability and validity of the scale as well as clarification of the relationship between RP and other crucial clinical variables. A second interesting direction could be to apply a patient self-report version of the RPPS-T to investigate the degree of convergence between therapist and patients’ scores and between them and the computerized linguistic RP measures. Another heuristic direction regards the correlation between RPPS-T and the therapist’s notes. Some recent studies (Bucci et al. 2012; Hoffman et al. 2013) found that the computerized linguistic RP measures applied to the therapist’s post-session reports are highly associated to the psychotherapy outcome. As for the therapist's notes, RPPS-T could be a reliable measure of what happens during the therapy process.

Conclusion

Taking into account the results of all three studies, we can consider the RPPS-T a useful instrument to provide an overall measure of the activation of RP in a session and especially of the symbolizing process. The new questionnaire received a first validation and represents a concurrent measure of the RP, additional to the method of computerized linguistic measures. It provides a method of investigating the RP from the point of view of the therapist, who has the unique position of being both directly involved into the therapeutic process and also having a conscious and professional vision of the processes that take place in it. Furthermore, having a less time-consuming self-report measurement of the RP is an advantage for monitoring the quality of the therapy process in daily clinical practice, in supervision and in research studies.

Table 1.

Pattern Matrix of the Retained Solution in the Exploratory Factor Analysis Conducted on 105 Administrations of the 42-item RPPS-T

Items	Factors
	Con/Im (1)	Ref/Reor (2)	Spe (3)	Mem (4)
1	.835	-.030	-.030	-.127
2	.814	-.030	.152	.104
3	.706	.074	-.148	.013
4	-.035	.956	-.044	.109
5	.177	-.739	.020	.092
6	.092	.666	.033	.016
7	.085	-.048	.968	.050
8	-.007	.033	.697	-.135
9	.187	-.029	-.483	-.021
10	.079	.161	.054	.879
11	.279	.146	.159	-.689
12	.231	-.152	.027	.603

Note. Principal axis factoring extraction method was used in combination with a promax rotation; in bold the factor loadings of the items retained in each factor; Con/Im = Concreteness and Imagery, Ref/Reor = Reflection and Reorganization, Spe = Specificity, Mem = Vividness of Session Memories.

Table 2.

Referential Process Post-session Scale – Therapist version (RPPS-T)

Factors	Items
(1) Concreteness and Imagery	1. We referred to the sensory, motoric and/or visceral aspects of feelings and emotions. 2. We have experienced significant bodily, motoric, perceptive and/or emotional sensations. 3. We used metaphors, images and/or situations to make more iconic what we were saying.
(2) Reflection and Reorganization	4. We connected among them symptoms, thoughts or events of the patient's life, understanding their meaning better. 5. We remained at a descriptive level, without deepening the meanings and explanations of what we were saying. (Reverse) 6. We did not reflect on the feelings, thoughts, and intentions behind the behavior of people. (Reverse)
(3) Specificity	7. We specified the details of what we were saying (people, times and places, features, ...). 8. We used verbal, abstract and decontextualized concepts. (Reverse) 9. We have been abstract and decontextualized in our speeches. (Reverse)
(4) Vividness of Session Memories	10. Now, after the session, the feelings and sensations felt during the session are still clear and well defined. 11. Now, after the session, I don’t have clear and emotionally vivid memories of what we lived during the session. (Reverse) 12. Now, after the session, I can make a precise and detailed picture of the experience lived together in the session.

Note. For the Italian version of RPPS-T please contact the corresponding author.

Table 3.

Pearson Correlations among the Four Factors of 12-item RPPS-T

	(1)	(2)	(3)	(4)
Concreteness and Imagery (1)	—
Reflection and Reorganization (2)	.599 ***	—
Specificity (3)	.623 ***	.649 ***	—
Vividness of Session Memories (4)	.646 ***	.691 ***	.716 ***	—

Note. *** p < .001.

Table 4.

Pearson Correlations between the Four Factors of 12-item RPPS-T and the Linguistic Measures of RP Computed on the Sessions Transcripts

RP linguistic measures	RPPS-T factors
	Con/Im (1)	Ref/Reor (2)	Spe (3)	Mem (4)	RA (1,3)
DfD	.336 *	.120	.141	.041	.270
RefD	-.265	-.026	-.137	-.066	-.226
SenSD	-.605 ***	-.547 ***	-.402 *	-.503 **	-.559 ***
WRAD	-.034	.137	.185	.315	.070
MHWRAD	.284	.383 **	.391 *	.444 **	.361 *
WRRL	-.492 **	-.286	-.365 *	-.316	-.473 **
MHWRRL	-.483 **	-.281	-.363 *	-.311	-.466 **

Note. * p < .05, ** p < .01, *** p < .001; Con/Im = Concreteness and Imagery, Ref/Reor = Reflection and Reorganization, Spe = Specificity, Mem = Vividness of Session Memories, RA = Referential Activity (mean of Con/Im and Spe); DfD = DisFluency Dictionary; RefD = Reflection Dictionary; SenSD = Sensory Somatic Dictionary; WRAD = Weighted Referential Activity Dictionary; MHWRAD = Mean High Weighted Referential Activity Dictionary; WRRL = Weighted Reflection and Reorganization List; MHWRRL = Mean High Weighted Reflection and Reorganization List.