Facilitative interpersonal skills are relevant in child therapy too, so why don't we measure them?

One of the consistent findings from psychotherapy process research has been the impact of therapist effects on patient change and the therapeutic alliance. The Facilitative Interpersonal Skills (FIS) paradigm is a task in which participants respond to standardized videos of actors playing patients in interpersonally challenging moments as if they were the therapist, which was designed to assess therapist effects. Participants' video recorded responses are coded for eight skills: verbal fluency, emotional expressiveness, warmth/acceptance/ understanding, empathy, persuasiveness, hope/positive expectations, alliance-bond capacity, and rupture-repair responsiveness. Performance-based procedures like the FIS minimize self-report bias and systematically control for client-related variability while maintaining strong clinical relevance. Research has shown that therapist FIS predicts the quality of the therapeutic alliance and outcome in adult psychotherapy. This paper describes the development and first adaptation of the FIS task using child and adolescent patients as the stimuli, and reports findings from a pilot study testing the reliability. The FIS-Child (FIS-C) task was administered to 10 therapists with a range of clinical backgrounds. Participants also completed the original FIS task and self-report measures of their empathy, social skills, and playfulness. Adequate interrater reliability was achieved on the FIS-C. There were no significant differences between participants' ratings on the FIS-C compared to the original FIS, although there were minor differences in the correlations between the FISC and self-report measures compared to the original FIS. Findings support moving forward with utilizing the FIS-C to empirically study therapist effects that may be common factors across treatment models.

Introduction

Although child psychotherapy research has lagged behind that for adults, multiple meta-analyses and systematic reviews now point to the effectiveness of both psychodynamic and cognitive behavioural approaches, as well as integrative treatments, for a wide range of mental and behavioural health challenges in children (Abbass et al., 2013; Pilling et al., 2020; Weisz et al., 2017). However, the effects of psychotherapies for youth have not increased or gotten stronger over the decades of research, suggesting that advances in treatment are needed (Weisz et al., 2017). When manualized treatments for youth populations are disseminated, they do not show superiority to treatment as usual (Weisz et al., 2013). The significance of relational skills that shape successful therapeutic relationships and interactions in child therapy may be particularly important to understand what drives treatment effectiveness (Karver et al., 2006; McLeod, 2011; Shirk & Karver, 2003; Halfon & Bulut, 2019). Indeed, metaanalyses suggest that the child-therapist alliance is related to outcomes (Karver et al., 2006; McLeod, 2011; Shirk & Karver, 2003; Halfon, Oszoy & Cavdar, 2019). But crucial questions remain, including: how do therapists promote a strong alliance with young patients, what therapist skills are related to outcomes, and what can be done to improve the effectiveness of treatments? One avenue to advance treatment is through research on shared elements or common factors of effective treatments for youth, particularly therapist relational skills, rather than adherence to manualized techniques. For robust research, we need research paradigms that can move this exploration forward. In this paper, we describe the development of the Facilitative Interpersonal Skills (FIS) for Child Therapy (FIS-C) paradigm, an adaptation of the FIS task used in psychotherapy process research broadly, and report on reliability and preliminary validation for this task and coding system.

Research on therapist effects

The impact of therapist effects on treatment outcomes has been one of the most important findings of the psychotherapy process research. Numerous studies reveal differences in treatment effectiveness, despite highly controlled designs and attempts to make treatment as standardized as possible (e.g. Anderson et al., 2009; Crits- Cristoph & Mintz, 1991; Luborsky et al., 1985; Rapley & Lodes, 2019). In other words, even when different therapists attempt to follow the same procedures and techniques for sessions, their patients still demonstrate variations in response to therapy and change. Most studies suggest that individual differences between therapists explain approximately 9% of the variance in therapy outcomes (Castonguay & Hill, 2017; Crits-Christoph & Mintz, 1991).

Therapist effects on outcome appear due, at least in part, to the interpersonal qualities of the therapist, such as empathy, hope and encouragement, and the nuances of the relationship between therapist and client (Anderson et al., 2009; Norcross, 2011; Safran & Muran, 2000; Solomonov et al., 2018). Therapist demographic characteristics, such as age, sex, training, or treatment orientation, do not significantly relate to observed variations in treatment effect (Okiishi et al., 2003). The two volume Psychotherapy Relationships That Work presents findings from the third interdivisional APA Task Force on Evidence-Based Relationships and Responsiveness, which identified the following elements of the relationship as demonstrably effective, effective or probably effective: alliance (in individual, child and adolescent, and couple and family therapy), collaboration, goal consensus, empathy, positive regard and affirmation, collecting and delivering client feedback, cohesion in group therapy, congruence/genuineness, the real relationship, emotional expression, cultivating positive expectations, promoting treatment credibility, managing counter transference and repairing alliance ruptures (Norcross & Wampold, 2018). The authors recommend that practitioners apply these aspects of evidence-based relationships alongside evidence-based treatments, and that training programs ‘provide competency based training in the demonstrably and probably effective elements of the therapy relationship’ (Norcross & Wampold, 2018, pg. 1897). They also recommend that researchers ‘[examine] systematically the associations among the multitude of relationship elements and adaptation methods to establish a more coherent and empirically based typology that will improve clinical training and practice’ (Norcross & Wampold, 2018, pg. 1898).

Facilitative interpersonal skills

A challenge to research on therapist relational skills is that they are difficult to isolate and measure, because therapists are influenced by patients. To operationalize and systematically assess therapist relational and interpersonal skills and behaviors, Anderson and colleagues (2009) developed a performance-based assessment called the FIS task.

In the FIS, the therapist or research participant is shown videos of actors playing patients in therapy. The vignettes give a brief description of the patient and then the patient speaks for about one minute. The scenarios are designed to be interpersonally challenging for the therapist to respond to. The participant watches each video one at a time, and when the video stops, the participant is told it is their turn to talk and they are to say something as if they were this patient’s therapist in this moment. Participants respond to multiple stimulus videos of different patients, and responses are video recorded and coded for the presence and quality of eight interpersonal skills: verbal fluency, emotional expression, empathy, warmth, acceptance and understanding, persuasiveness, hope and positive expectations, alliance bond capacity and rupture repair responsiveness. These skills were selected based on the clinical and research literature identifying therapist qualities theorized to help facilitate the therapeutic process and the alliance, including those reported on in meta-analyses conducted and referenced by Norcross over the years (Anderson et al., 1999).

An overview of each code is provided in Table 1. Several of the skills are reflective of Rogerian humanistic theories that the therapist’s ability to provide empathy, warmth, and positive regard is a necessary first step in creating an alliance (Rogers, 1957). They also draw on years of psychotherapy research demonstrating the importance of addressing and negotiating ruptures in the therapeutic alliance (Safran & Muran, 2000). Verbal fluency and emotional expressiveness are rooted in Laura Rice’s research on vocal quality (Rice & Kerr, 1986). Persuasiveness refers to the ability to create a shared understanding and stems from Frank & Frank’s Persuasion and Healing (1975). And instilling hope and addressing expectations are components of evidence-based treatments (Constantino et al., 2011).

Table 1.

Descriptions of the 8 facilitative interpersonal skills.

FIS Code	Description
Verbal Fluency	Rates the participant’s level of comfort and ease in communicating with the patient
Hope and Positive Expectations	Rates expressions of hope, optimism, and positive expectations for change. When working with children, hope and positive expectations can be expressed in the tone of playfulness or themes of the play, rather than expressed explicitly
Persuasiveness	Rates the capacity to express a clear, organized understanding about the meaning of the patient’s source of distress. For children, ratings consider whether the point of view is communicated in a developmentally ap- propriate manner depending on the patient’s age
Emotional Expression	Rates the level of emotionality and energy in the participant’s response. For children, emotional expression should be attuned to the child’s level of engagement and emotional regulation
Warmth, Acceptance and Understanding	Rates the participant’s demonstration of care, acceptance, and understanding of the patient
Empathy	Rates the degree to which the participant expressed an accurate understanding of the patient’s thoughts, emotions or internal experience
Alliance Bond Capacity	Rates the extent to which the participant creates a collaborative environment or a sense of ‘we’
Alliance Rupture Repair Responsiveness	Rates the extent to which the therapist has a clear appreciation for the interpersonal problem presented and the feelings in the room and attempts to address or ‘heal’ those feelings

Studies using the FIS have demonstrated that the quality of therapists’ skills on the FIS task is significantly related to patients’ treatment outcomes (Anderson et al., 2009; 2016). Therapists with higher FIS scores saw patients who had better outcomes than patients seen by therapists with low-FIS and demonstrated greater rates of change on the Outcome Questionnaire-45 (OQ-45; Lambert et al., 2004) (Anderson et al., 2009; Anderson et al., 2016). Patients treated by high-FIS therapists also reported stronger therapeutic alliances after the first session and greater improvements in alliance over the course of treatment, compared to those seen by low-FIS therapists (Anderson et al., 2016). Thus, therapists’ FIS have a significant bearing on both patient outcome and perception of their relationship with the therapist. Yet, such a tool does not exist for research on psychotherapy with children and adolescents.

Relational factors and therapist effects in child psychotherapy

There are an increasing number of qualitative and quantitative studies looking at markers of good process in child therapy, pointing to the importance of relational aspects of treatment for children and adolescents (see for example the Special Issue on Qualitative and Quantitative Research in Child and Adolescent Psychotherapy; Salcuni, Capella & Lis, 2015). The most recent meta-analysis of therapeutic relationship variables in child and adolescent treatment identified 16 treatment process studies looking at therapist interpersonal skills in relation to outcome (Karver et al., 2006). Across studies, therapists’ interpersonal skills were correlated with a number of therapeutic process variables, such as attendance, alliance and engagement (Karver et al., 2006). Here we report research supporting aspects of the therapeutic relationship in child therapy, which we suggest a child FIS task could help expand.

Alliance. Collaboration between therapist and patient appears to facilitate therapeutic change and the alliance with youth (Creed & Kendall, 2005; Karver & Caporino, 2010; Karver et al., 2008). Introducing treatment clearly and with a comprehensible rationale is strongly related to both treatment participation and outcome (DiGiuseppe et al., 1996; Karver et al., 2006), and continued collaboration on personal goals over the course of treatment relates to improvements in the therapeutic alliance (Diamond et al., 1999, p. 364). In qualitative interviews, children in therapy noted that the ability to talk to a therapist, the therapeutic relationship and the space to play were most central to their positive change (Capella et al., 2018). Attending to what was happening in the therapeutic relationship, including talking about and processing the relationship, is considered by therapists across orientations to be the most important factor in promoting the alliance with adolescent patients (Groth & Hilsenroth, 2021).

Rogerian principles. Consistent with humanistic approaches, therapists’ unconditional positive regard and attention to the patient is related to improvements in the therapeutic alliance with youth (Diamond et al., 1999). Empathic resonance contributes to therapy outcomes with adults (Howard & Orlinsky, 1986), and validation has been found to relate to patient engagement and alliance ratings in youth treatments (Creed & Kendall, 2005; Karver et al., 2008). Children and adolescents need to feel not only accurately heard, but also accepted, as they may not always be accepted with other authority figures in their life. Genuineness and authenticity are key, though. Creed and Kendall (2005) found that therapists’ attempts to relate to or ‘find common ground’ with the child were negatively related to early ratings of the alliance (p. 504). They suggest that youth patients may perceive the therapist’s trying to find commonalities with them as disingenuous efforts to connect with them, or as ‘trying too hard’ (Creed & Kendall, 2005, p. 504).

Rupture-Repair. Rupture-repair processes are associated with better outcomes for adult patients, but research with youth is minimal. Data does confirm that adolescents experience both withdrawal and confrontation ruptures, as well as rupture sessions or phases throughout treatment (Schenk et al., 2019). Common repair strategies with adolescents mirror the relational elements identified by Norcross and others, namely: inviting thoughts and feelings, validating defensiveness, and providing a rationale for tasks and treatment (Schenk et al., 2019). While there are no studies we are aware of that look at rupture and repair with younger children, Nof and colleagues (2019) proposed a general Child Alliance Focused Approach, based on the empirical literature. In this approach, the repair process includes the therapist pausing to attend to the moment of the rupture, reflecting on and understanding the underlying meaning, identifying the rupture with the child, accepting responsibility and avoiding blame, framing the rupture in terms of ‘distress signals and a longing for’ protection and connection, and using change strategies and metacommunication tailored to the child’s developmental level (Nof et al., 2019).

Mentalization. There is a growing body of research on youth psychotherapy process using the Child Psychotherapy Q-Sort (CPQ; Schneider & Jones, 2004). Findings from research on a psychodynamic child treatment showed that child-centered aspects of the relationship significantly contribute to change among children with externalizing behaviors (Halfon et al., 2020). Adhering to principles of mentalization was also associated with changes in affect regulation and symbolic play over the course of therapy among children with externalizing behaviors (Halfon & Bulut, 2019). Characteristics of mentalizing processes in therapy align with the FIS, and include the therapist being sensitive to the child’s feelings, accurately perceiving the therapeutic process, emphasizing verbalization of internal states, being sensitive to the child’s level of development, creating a shared vocabulary or understanding, commenting on changes in mood or affect, exploring relationships, making links between feelings and experience, and addressing interruptions or breaks in the treatment. Qualities that are opposed to this process include being judgmental, nonresponsive, or reassuring.

Playfulness. One relational skill with children that has not been a focus in work with adult patients is the level of playfulness in the therapeutic environment. According to therapists interviewed by Campbell and Simmonds (2011), the therapeutic space is ‘the child’s world’ and should be like ‘a wonderland’ (p. 202), allowing the child to leave the outer world and providing a space to contain their deep emotions. One therapist described the need for the therapist themselves to experience their own ‘inner child, the joy of the therapeutic surrounds… a sense of fun and delight’ (Campell & Simmonds, 2011 p. 203). Another therapist stated that when the therapy room itself lacks resources, he relies on his creativity and imagination in order to create the feeling of a ‘wonderland’ (Campell & Simmonds, 2011).

Adapting the FIS for child therapy: creating the FIS-C

Research on relational processes with children is growing, but there is no tool like the FIS that allows operationalization and assessment of therapist interpersonal qualities and behaviors that promote change. Given that child psychotherapy differs significantly from adult psychotherapy, it is crucial that assessments of therapist qualities and relational skills be specific to child therapy. Our research group developed the FIS-C to address the need for a set of standardized stimulus videos to systematically evaluate therapists’ skills, as well as a set of operational skills on which to focus.

FIS-C stimuli

The FIS-C stimuli were developed in the Attachment & Psychotherapy Process Lab at Ferkauf Graduate School of Psychology, with two actors (8 year-old male and 12 year-old female) portraying child clients. Similar to the original FIS videos (Anderson et al., 2018), the vignettes feature an interpersonal demand or relational theme and reflect a range of interpersonal styles from the Interpersonal Circumplex Model, from hostile to friendly and dominant to submissive (Di Blas, Grassi, Luccio, & Momente, 2012). Three distinct types of alliance ruptures are represented in the FIS-C videos: confrontation, withdrawal, and mixed ruptures. The interpersonal styles represented within the child stimulus clips include the following patient profiles: i) confrontational patient (hostile, dominant); ii) passive and withdrawn patient (hostile, submissive); iii) yielding patient (hostile, friendly); and iv) dominant and controlling patient (friendly, dominant). A total of 7 vignettes were made. In keeping with the FIS Task and Coding System Manual (Anderson et al., 2018), the FIS-C manual includes short paragraphs about the interpersonal demands and potential components of a meaningful resolution for each scenario, to guide the coding.

An initial pilot of the child clips was conducted with child clinicians in a hospital setting who volunteered for a training in interpersonal and relationship skills in child therapy. The authors, along with two additional doctoral students at Ferkauf Graduate School of Psychology who were trained in the FIS Coding System, used these responses to determine whether the eight FIS were applicable with modifications to the codes to capture how the skills may appear in work with children. For example, there is less reliance on verbal content in many of the codes because language used with children and adolescents is necessarily simpler.

Current study

The purpose of the current study was to examine the reliability and validity of the newly developed FIS-Child Task and Coding System by administering and rating the FIS-C alongside the original FIS-Adult or FIS-A and self- report measures of empathy, social skills, and playfulness. We hypothesized that (H1) the stimulus clips created for the FIS-C could be reliably rated for the same set of interpersonal skills represented in the original stimulus clips of the FIS (Anderson et al., 2018). Second, we predicted that (H2) there would not be significant differences between mean FIS ratings on the FIS-C videos compared to the original FIS-Adult, and that there would not be significant differences in ratings on the individual scales (e.g., verbal fluency, empathy, etc.). Third, we hypothesized that (H3) there would be differences in the FIS-C and original FIS-Adult or FIS-A ratings depending on whether the participant identified as a child therapist, such that people who identify as child therapists would have higher mean FIS ratings on child focused clips, and those who identify as adult therapists would have higher ratings on the adult focused clips. In terms of relationships with self-report measures, we predicted that (H4) the relationships between performance on the FIS-C and self-report measures of empathy and social skills would be similar to the findings from the FIS-A studies, such that any associations would be small. Lastly, we hypothesized that (H5) child psychotherapy work would involve a greater level of playfulness as compared to adult psychotherapy work, and thus ratings of the FIS-C would be associated with a self-report of adult playfulness.

Materials and methods

Participants

All participants were mental health clinicians, either licensed or in training (n=10; 9 female, 1 male). Participant age ranged from 24-50 years old (mean=29.50, SD=8.07). Most participants (n=8) were currently in training pursuing a doctoral degree in Psychology (Ph.D. or Psy.D.), one participant had already completed their doctoral training, and one held a professional license in social work.

In terms of race/ethnicity, six clinicians identified as White, one identified as African-American/Black, and three identified as Asian/Asian-American. No participants in the current sample identified as multiracial or biracial.

In terms of primary treatment orientation, two participants identified as Cognitive/CBT, two as Psychodynamic or Psychoanalytic, one as Interpersonal, one as Feminist, one as Systemic, and three identified ‘Not Sure’ of their primary orientation. Four participants reported that they work with both children and adults in clinical settings, one reported working only with adults, and two reported working only with children and adolescents. Three participants did not indicate the age of participants with whom they primarily work. Participants were also asked to indicate how many years of experience they have working in the field. Two participants reported having one year of experience, three had two years of experience, one indicated having three years of experience, one had five years of experience, two had 6-10 years, and 1 reported over 25 years of experience.

Procedures

This study was approved by the WCG Institutional Review Board for Yeshiva University. Both trainee and licensed psychotherapists were recruited to participate via emails to professional and academic listservs, including Section 2 (Childhood and Adolescence) of APA Division 39 (Society for Psychoanalysis and Psychoanalyic Psychology), graduate schools in the New York metropolitan area, Early Career Therapists of New York, and APA Division 29 (Society for the Advancement of Psychotherapy).

Clinicians who were interested were directed to a Qualtrics form, where informed consent was obtained before continuing. Both the performance task and self-report aspects of the study were completed online. Participants could complete the survey in more than one sitting but were required to complete the entirety of the video performance task within one sitting. Those who agreed to participate were given instructions to complete the FIS task and record their responses.

Participants watched and responded to seven FISChild videos and four FIS-Adult videos, which were displayed in a random order. They then completed self-report questionnaires. For this study, we will be reporting on results related to demographic background information, empathy, social skills, and playfulness.

Measures

Facilitative interpersonal skills

The FIS Coding System (Anderson et al., 2018) rates participants’ responses to the stimulus clips on a set of eight codes. Each code is rated on a 5-point Likert-type scale, ranging from ‘1’ (Not Characteristic) to ‘5’ (Extremely Characteristic). The scale can be used as a continuum and thus half-points are allowed. Specific descriptions of each rating level for each quality are provided within the manual. In rating a response, instances of an ‘average’ level of a helping behaviour would merit a rating of 3. A rating of 3 is ‘considered the default rating for all items and are considered ‘ordinary’ helping or facilitative interpersonal skills’ (Anderson et al., 2018, p.16). Individual responses are assigned a code for each of the 8 skills, as well as a mean rating from all 8 skills for that stimulus clip. The mean of all codes across all videos is considered the therapists’ FIS score and used in analyses. Separate means were calculated for the FIS-C responses and the FIS-A responses.

Responses to the original FIS and the FIS-C performance task were coded separately by the two authors of this paper. The authors met weekly to discuss coding and reach consensus when codes differed by more than 1 point on the Likert scale. Inter-rater reliability is reported in the results section.

Self-report measures

Interpersonal Reactivity Index (IRI). The IRI is a 28- item self-report questionnaire assessing four subscales of the broad concept of empathy, including perspectivetaking and ‘feelings of warmth, compassion and concern for others’ (Davis, 1980, p. 2). IRI items are summed to create a total score ranging from 0 to 112. In the current study, one item was left out of the questionnaire in error (Item 8), typically included in the Perspective Taking (PT) subscale. For the purposes of the current study, the mean of the participants’ PT subscale score was used in place of Item 8, in order to use this scale within the pilot context. The IRI has demonstrated acceptable psychometric properties (Davis, 1980). In the current sample, reliability for the total score was low (=0.424), while the four subscales demonstrated acceptable reliability (Perspective Taking, =0.824; Fantasy, =0.719; Empathic Concern, =0.551; and Personal Distress, =0.785). Anderson and colleagues (2016) demonstrated construct validity to support that the FIS accurately taps into therapists’ level of empathy, with FIS scores moderately correlating with therapists’ scores on a similar measure of empathy (California Psychological Inventory). However, Perlman and colleagues (2020) recently found that the FIS was not significantly related to self-reported empathy measured by the IRI.

Brief Social Skills Inventory (BSSI). The BSSI is a 30-item self-report scale assessing 6 domains that underlie social competence. Participants use a 5-point scale to indicate the extent to which each item applies to them, and scores represent one’s ability to send, receive, and manage information in social and emotional domains. Research demonstrates that the BSSI serves as a valid and reliable measure of emotional intelligence (Riggio, 1986). Scores on the BSSI total range from 0 to 150, and each subscale can range from 0 to 25. In the current sample, the overall BSSI demonstrated acceptable reliability (=0.681). Cronbach’s alpha for the six subscales were as follows: Emotional Expressivity, =0.107; Emotional Sensitivity, =0.771; Emotional Control, =0.801; Social Expressivity, =0.956; Social Sensitivity, =0.784; Social Control, =0.779). Anderson et al. (2015) found that the full-length 90-item Social Skills Inventory (SSI) significantly predicted symptom reduction across sessions as well as therapist reports of the alliance.

Adult Playfulness Trait Scale (APTS). The APTS is a 28-item self-report questionnaire that has been developed and demonstrated to effectively differentiate adults with different degrees of playfulness, including constructs of personality, behaviour, attitude and perception (Shen et al., 2014a; Shen et al., 2014b). In a sample of 209 adults, Shen et al. (2014b) found a mean APTS score of 4.78. The APTS is intended to tap into three distinct areas of playfulness: Fun Seeking Motivation (the extent to which one seeks to have fun in all situations), Uninhibitedness (the ability to act without restraint), and Spontaneity (the tendency to act impulsively without thought). In the current sample, the Fun Seeking Motivation and Spontaneity subscales demonstrated strong internal reliability (=0.874 and =0.938, respectively), while the Uninhibitedness scale demonstrated low reliability (=0.479).

Data analytic plan

The original FIS (FIS-Adult or FIS-A) and FIS-C were both coded by two independent coders who were blind to the self-report data. A mean FIS score was calculated for both the FIS-A and FIS-C. Single-measures ICCs were calculated to assess inter-rater reliability for the FIS-C and FIS-A ratings on individual scales and the mean score (H1). Cronbach’s alphas were calculated to assess the internal consistency of the 8 FIS scales (H1). Differences between ratings on the FIS-A and FIS-C were tested using paired samples t-tests (H2). An independent samples t-test was conducted to compare the FIS-C scores of therapists who identified working with children in their clinical practice versus those who do not work with children (H3). Given the small sample size, Spearman’s Rho correlations were used to assess whether participants’ age related to performance on the FIS (H3). To assess whether therapists’ amount of clinical experience related to performance on the FIS, an independent samples t-test was conducted to compare the FIS scores of participants with fewer than 5 years of clinical experience with those with 5 years or more of experience (H3). Finally, Spearman’s Rho correlations were used to assess the correlations between the FIS-A and FIS-C and subscales of the IRI (empathy; H4), BSSI (social skills; H4), and APTS (play; H5).

Results

Table 2 reports descriptive data from the FIS-Adult, FIS-Child, and self-report measures.

Table 2.

Means and standard deviations for FIS-A, FIS-C and self-report measures.

	M	SD	Range	Cronbach’s alpha
FIS-Adult	3.62	0.21	3.22-3.86	0.90
FIS-Child	3.45	0.21	3.08-3.73	0.92
Interpersonal Reactivity Index (IRI)	69.65	6.01	60.0-80.0	0.424
Perspective taking	22.43	3.63	18.7-28.0	0.824
Fantasy	17.55	4.44	12.0-24.0	0.719
Empathic concern	22.77	2.64	17.0-26.0	0.551
Personal distress	6.88	4.68	2.0-14.0	0.785
Brief Social Skills Inventory (BSSI)	100.22	10.60	85.0-116.0	0.681
Emotional expressivity	16.56	2.12	14.0-21.0	0.107
Emotional sensitivity	18.56	3.39	14.0-24.0	0.771
Emotional control	19.00	3.74	15.0-25.0	0.801
Social expressivity	14.11	6.13	5.0-23.0	0.956
Social sensitivity	15.56	3.61	11.0-20.0	0.784
Social control	16.44	3.97	12.0-21.0	0.779
Adult Playfulness Trait Scale (APTS)
Fun seeking motivation	4.92	0.59	3.88-5.75	0.874
Uninhibitedness	3.64	0.79	2.60-5.20	0.479
Spontaneity	2.91	1.18	1.00-4.60	0.938

M, mean; SD, standard deviation.

Hypothesis 1: Reliability of ratings of facilitative interpersonal skills

Interrater reliability was adequate (ICC=0.763) for the mean FIS ratings on both the FIS-A and FIS-C. In following the guidelines of Cicchetti & Sparrow (1981), results also demonstrated high interrater reliability for the individual codes of Empathy and Alliance Rupture Repair Responsiveness, and acceptable reliability for the remaining 6 codes. Interrater reliability for each scale can be found in Table 3.

Table 3.

Interrater reliability for FIS-Child Codes.

FIS Code	ICC
Mean FIS	0.763
Verbal fluency	0.691
Hope and positive expectations	0.660
Persuasiveness	0.661
Emotional expression	0.683
Warmth, acceptance, understanding	0.612
Empathy	0.770
Alliance bond capacity	0.657
Alliance rupture repair responsiveness	0.783

ICC, intra-class correlation coefficient.

Cronbach’s alpha was calculated based on the eight codes that comprise both the original FIS-A and the FISC (=0.90 and =0.92, respectively). As such, statistical analyses support the prediction that the eight qualities that comprise the FIS-C represent a unitary construct. These results are consistent with those of Anderson and colleagues (2018), who similarly found that the eight items of the FIS represented a single construct and were internally consistent. Given the high internal reliability and interrater reliability, the mean scores were used as the operational representation of FIS-C performance in all other statistical analyses in this study.

Hypothesis 2: Ratings on FIS-Child compared to FIS-Adult

There was not a significant difference found between the mean FIS-C scores (M=3.45, SD=0.21) and the mean original FIS-A scores (M=3.62, SD=0.21). Overall performance in response to the child stimulus clips is relatively similar to therapists’ performance on the adult stimulus clips.

Hypothesis 3: FIS-Adult and Child ratings and therapist demographics

FIS-A and FIS-C ratings were not related to either age or years of experience. An independent samples t-test comparing the FIS-C scores of therapists who reported working with children in their clinical practice versus those who do not work with children showed that there was not a significant difference in either FIS-A or FIS-C scores between those who work with children and those who work only with adults.

Hypothesis 4 and 5: Correlations between FIS-C and self-report scales - Empathy, Social Skills, Playfulness

Spearman’s Rho correlations were used to assess the relationships between self-reported personality traits or qualities and performance on the FIS-A and FIS-C. One participant completed the FIS performance task without completing the self-report questionnaires, so the subsequent data discussed includes nine participants (n=9).Correlations can be found in Table 4.

Table 4.

Spearman’s Rho correlations between performance on the FIS and self-report measures of interpersonal qualities.

	FIS-Child	FIS-Adult
Interpersonal Reactivity Index (IRI) Total	0.310	0.544
Perspective taking	0.017	0.152
Fantasy	0.084	0.143
Empathic concern	0.319	0.664
Personal distress	0.000	0.085
Brief Social Skills Inventory (BSSI) Total	–0.550	–0.200
Emotional expressivity	0.342	–0.530
Emotional sensitivity	–0.300	–0.183
Emotional control	–0.151	0.101
Social expressivity	–0.824**	0.092
Social sensitivity	0.114	–0.035
Social control	–0.093	–0.152
Adult Playfulness Trait Scale (APTS)
APTS - Fun seeking	–0.812**	–0.042
APTS - Uninhibitedness	–0.276	0.050
APTS - Spontaneity	–0.762*	–0.293

*P≤0.05; **P≤0.01.

A positive correlation was found between the FIS-A scores and the Empathic Concern subscale of the IRI, which approached significance (r(7)=0.664, P=0.051). This indicates that participants who self-reported a higher degree of empathic concern also demonstrated higher FIS scores in response to the adult stimulus clips, compared to those with lower empathic concern. No significant correlations were found between the IRI overall score or subscales and the FIS-C.

A significant negative correlation was found between the FIS-C scores and the Social Expressivity subscale of the BSSI, (r(7)= –0.824, P=0.006). The Social Expressivity scale taps into skills in verbal communication and ability to engage with others during social interactions. Participants who endorsed greater agreement with items of social expressivity tended to score lower on the FISC. No other subscales within the BSSI resulted in significant correlations with scores on the FIS-C. No significant correlations were observed between subscales of the BSSI and performance on the FIS.

Significant negative correlations were found between scores on the FIS-C and the Fun Seeking Motivation subscale of the APTS (r(7)= –0.812, P=0.008. FIS-C scores were also significantly negatively correlated with the Spontaneity subscale of the APTS (r(7)= –0.762, P=0.017). As such, participants who endorsed a greater degree of seeking fun in most activities and those who endorsed being more spontaneous, performed with lower scores on the FIS-C.

Discussion

The primary aim of this pilot study was to examine the reliability and potential validity of a child-therapy adaptation of the FIS Task and Coding System (Anderson, 2013). The FIS-C was created to operationalize and systematically assess therapist effects that may explain treatment outcomes with youth. Anderson and colleagues (e.g., 2009; 2016; 2018) have found strong evidence to support the FIS as a reliable and valid measure representative of therapists’ interpersonal skills that positively relate to improved treatment outcomes and a stronger therapeutic alliance with adult patients. The goals of this study were to assess whether the same set of interpersonal skills can be reliably rated in simulations of child psychotherapy, and to examine the relationship between therapists’ FIS in child therapy and their self-reported interpersonal tendencies, such as empathy and social skills.

Due to challenges with data collection, these early data are limited to a sample of 9-10 clinicians, 8 of whom were trainees and only one of whom had more than 20 years of clinical experience. Thus, the empirical findings must be interpreted with a great deal of caution. Nevertheless, as there is currently no trans-theoretical, standardized assessment that we are aware of for rating therapist interpersonal skills with children and adolescents, these data are encouraging for continuing development and evaluation of the FIS-C.

The first research question was whether the eight FIS were applicable to and could be reliably rated within the context of child and adolescent psychotherapy. Adequate interrater reliability (Cicchetti & Sparrow, 1981) was found for each of the eight codes of the FIS-C in this sample, where seven responses were rated for each of the 10 therapists. Additionally, high internal reliability was found across the eight codes, providing preliminary support for the prediction that these qualities represent one unitary FIS construct rather than eight distinct constructs. This is consistent with the results of Anderson and colleagues for the original FIS (2009).

Our sample of therapists performed similarly in response to both the child stimulus clips and the adult stimulus clips, suggesting that the child-focused adaptation of the FIS is consistent with the original measure and that therapists tend to use the same general set of interpersonal skills with young patients as they do with adults. Given the similar performance across both sets of stimuli, we are hopeful that the FIS-C has validity in capturing the interpersonal skills it is designed to measure.

We also found non-significant results when investigating whether performance on the FIS-C was impacted by therapist’s years of clinical experience and whether they have experience providing therapy to youth, which may be due to the small and limited sample and needs to be explored further. Participants who identify as working primarily with children versus primarily with adults performed statistically equivalently on the child clips and the adult clips. We had predicted that therapists who work with youth in clinical practice would have higher scores on the FIS-C as compared to therapists who work primarily with adults, given any specialized training and general interest in working with children. Once again, the lack of significance may be specific to this sample or due to small sample size. However, it may be that performance on the FIS is not significantly impacted by previous training or experience working with youth, and that generally speaking, therapists may be able to adapt their interpersonal skills to the developmental level of their patient.

Participants’ age and level of experience was also not significantly related to performance on the FIS. This finding is somewhat contradictory to findings of Anderson et al. (2009), which demonstrated that age and FIS performance were significantly positively correlated. Anderson et al. (2009) argued that the set of interpersonal qualities that ultimately promote therapeutic change take continuous effort and practice, and thus may inherently improve with increased age and clinical practice. Our sample was very limited in terms of the range of experience, which may explain the lack of significance, and this question should be further explored in a larger sample size with adequate power. Most participants in the current sample were still in clinical training, and only one participant reported having more than 20 years of experience.

Therapists’ self-reported empathy was not significantly positively related to their FIS performance on the stimulus clips (Empathic Concern approached significance with regard to the adult clips). While empathy is a necessary therapeutic skill, the performance of empathy may differ from self-reported empathy, as a trait. It is possible that the IRI does not tap into the type of empathy that is expressed by therapists. Future studies should continue to investigate whether the FIS-C captures therapists’ use of empathy when working with children and adolescents, in larger samples and perhaps considering other measures.

The significant findings relating to the Adult Playfulness Trait Scale have important implications for therapists working with children in youth psychotherapy, though not necessarily the understanding of child therapy that we had intended to capture. The Fun Seeking Motivation subscale and the Spontaneity subscale were both significantly negatively correlated with performance on the FIS-C. The APTS Fun Seeking Motivation subscale assesses one’s internal motivation to derive fun from their environment and includes items that measure one’s beliefs about prioritizing enjoyment in life, efforts to seek fun activities, and being reactive to fun stimuli (e.g. ‘I believe fun is a very important part of life;’ ‘I can find fun in most situations;’ Shen et al., 2014). Participants who strongly endorsed having fun-seeking beliefs and motivation scored lower on the FIS-C performance task. This measure was included in the current study because of clinical observations and support in previous literature that working with children in psychotherapy requires a certain degree of ‘playfulness’ and creativity (e.g. Campbell & Simmonds, 2011). Thus, it was predicted that high scores on the APTS Fun-Seeking Motivation subscale would be positively correlated with performance on the FIS-C. Upon further consideration of the results, it can be interpreted that clinicians who rate themselves as seeking out and prioritizing fun in all situations may be prioritizing having fun above therapeutic work. As any therapist can attest, therapy sessions are not necessarily always fun, particularly during difficult moments when a patient of any age is dysregulated or in crisis. Furthermore, although child therapists often engage their clients through play, the aim of therapy sessions is not merely to have fun and focus on positive feelings. Prioritizing having fun in all situations may, at times, be detrimental to the therapeutic relationship and important therapeutic work. While informative nonetheless, this scale appears to measure a domain of ‘playfulness’ that differs conceptually from the type of playfulness that we believe to be relevant and necessary in psychotherapy. In line with Campbell & Simmonds (2011), therapy sessions with children are imaginative, creative, and flexible, often involving a variety of toys and games, prompting the therapist to tap into their ‘inner child,’ and allowing the child to leave their everyday world. This does not necessarily mean that the therapy sessions themselves are always ‘fun,’ as they may still stir up negative emotions or difficult memories. Future research should investigate the FIS-C in relation to other measures of playfulness that are more reflective of creativity and imagination, rather than seeking to have fun.

Additionally, the APTS Spontaneity subscale assesses one’s tendency to respond to stimuli promptly or spontaneously without significant forethought (e.g. ‘I often do unplanned things;’ ‘I often act on my impulses;’ Shen et al., 2014a). It was similarly predicted that higher ratings of spontaneity would relate to higher scores on the FISC, given the notion that therapy, particularly with children, requires a degree of flexibility and ability to adapt (Campbell & Simmonds, 2011). The negative correlation found could be interpreted, however, such that therapists who endorse that they often act on impulses may respond to the FIS-C stimulus clips impulsively without thinking through their response fully and without self-awareness. While many therapists may endorse that working with children necessitates flexibility and spontaneity, therapeutic work inherently requires thoughtfulness from the therapist. As Anderson and colleagues (1999) describe, facilitating a strong therapeutic relationship involves continuous, moment-to-moment attunement to the patient’s verbal and nonverbal behaviors. While this certainly involves flexibility and adaptation, it cannot be done impulsively or without thought. In thinking about these results, future research would benefit from an alternative measure that more accurately taps into the type of flexibility and creativity involved in psychotherapy.

In general, the lack of significant correlations between the FIS-C or FIS-A and the Brief Social Skills Inventory suggests that therapists’ interpersonal skills are not the same as self-reported social skills. However, we observed a significant negative correlation between FIS-C scores and the Social Expressivity subscale. The Social Expressivity subscale measures one’s propensity to communicate verbally and nonverbally and engage with others during social interactions. Thus, high scores on this subscale indicate that a participant views themselves as fairly outgoing and extroverted, in a sense. We observed that greater levels of Social Expressivity on the BSSI were significantly positively correlated with greater levels of Fun Seeking Motivation on the APTS. In other words, those who rate themselves as highly outgoing and sociable also endorse that they seek and prioritize having fun in most situations. Together, these two traits seem to point to an extroverted or outgoing personality type that may be negatively correlated with the facilitative interpersonal skills. This implies that in the context of therapy sessions, it is not inherently that the more outgoing a therapist is, the better.

Limitations

The primary limitation of this study is the small sample size. The COVID-19 pandemic presented unanticipated challenges with recruitment. Our study requested online participation for up to an hour during a time when most people reported feeling burnt out by screen time and under a great deal of stress generally. Furthermore, eight out of ten participants were in training, and only two were licensed clinicians. Only one of the participants had more than 25 years of experience. As such, the findings discussed must be considered with caution. Future research is necessary with larger sample sizes including a broader range of experience to make conclusions about the utility and validity of the FIS-C.

Second, this study relied on self-report for the measures of social skills, empathy and playfulness. Self-report is subject to biased or inaccurate reporting. Future research should investigate correlations between performance on the FIS-C and more objective measures of the traits investigated herein. We also found, upon closer inspection, that the Adult Playfulness Trait Scale appeared to measure domains of the concept of playfulness that differ from the type of playfulness we conceptualize as relevant to psychotherapy with youth and that we had initially intended to capture, implying a potential need for an alternative measure of playfulness.

Lastly, due to the nature of recruitment, this study is subject to self-selection bias, such that those who participated chose to do so out of their own interest. The results may have been impacted by being made up of participants who may have an inherent interest in the subject matter.

Clinical implications and future directions

Despite the small sample size and limitations of this study, the findings support moving forward with further evaluations of the FIS-C to empirically study therapist effects and interpersonal skills that may be common factors across treatment models. Such a tool has significant implications both in understanding the processes through which change occurs and the skills to incorporate into training of future clinicians. Future research should continue to assess the validity of the FIS-C by investigating possible correlations with other measures of the same skills and qualities and with larger sample sizes. A critical next step in this work will be to assess the relationship between FIS-C performance and therapy outcomes with children and adolescents.

Being able to operationalize and reliably observe a distinct set of interpersonal skills that promote therapeutic change has the potential to add to our understanding of the processes through which psychotherapy works, with implications for training. Most recently, Anderson’s research (2020a,b) has shown that certain forms of experiential training and practice may help therapists improve in these skills, which could have a positive effect on their patient care. There may be benefits to training targeting the interpersonal qualities captured by the FIS. Evers and colleagues (2019) propose that tailoring training to trainees’ developmental level, with more integration of didactic and practical elements as well as deliberate practice may have a positive impact on trainees’ experiences of healing and stressful involvement over their training. Current training programs typically focus on treatment modalities and content of therapeutic work, and this body of research implies that there may be utility in teaching these more nuanced interpersonal skills, through experiential and deliberative practice techniques. There are in fact multiple models for training in foundational helping skills in adult psychotherapy (Hill & Lent, 2006), which are similar to the FIS, but there is only one study to date on helping skills in child therapy (Schechtman, 2004). If future research could link the FIS-C to outcomes in child therapy, we may have a set of concrete, operationalized interpersonal skills to focus training for child therapists across various treatment modalities and orientations. Furthermore, the FIS-C may provide a structure for training that is focused on embodied aspects of child psychotherapy, in line with an embodied learning approach that has been proposed (Gennaro et al., 2019).

We must acknowledge that video tasks like the FIS are merely simulations and there is a great deal that they do not include. For example, in real life therapeutic interactions there is a back and forth, therapists can see their patient’s nonverbal or verbal responses to their interventions as they speak and change course or more closely attune. With little background information about the patient and no history together, the FIS does not necessarily involve the ability to form and utilize a conceptualization. It does however capture therapists’ in-the-moment interpersonal and relational skills. While the face validity of the task may be questioned by some participants, the empirical evidence suggests that the FIS is at least an adequate proxy for therapists’ interpersonal skills in practice. Standardization allows us to compare therapists with each other in ways that are for the most part not possible using data from therapy sessions. For the field to continue to move forward in understanding the mechanisms of action in child therapy, a range of research methods will be necessary. The FIS-C is one such tool that may allow us to advance child psychotherapy research.