Becoming Trauma Informed: Validating a Tool to Assess Health Professional's Knowledge, Attitude, and Practice.

To effectively address the negative health effects of early childhood trauma and adversity, healthcare professionals and healthcare institutions must understand the impact of adverse childhood experiences and trauma on health. This study aimed to validate a tool to assess knowledge, attitude, and practice of trauma-informed care among interdisciplinary pediatric healthcare staff.
METHODS: A 36-item survey tool, "Knowledge, Attitudes, and Practices of Trauma-Informed Practice," was adopted and modified with permission from the author. We administered the survey electronically to 2,659 staff at a pediatric healthcare institution. To assess the tool's reliability and validity, internal consistency reliability testing, content validity, and construct validity assessments were conducted.
RESULTS: A total of 592 surveys were collected, representing a 22.3% response rate. Confirmatory factor analysis revealed that 21 items provided the strongest internal consistency reliability for the overall tool and each factor. The overall Cronbach's alpha for the 21-item tool was 0.86, with 0.84 for the knowledge factor, 0.74 for the attitude factor, and 0.78 for the practice factor. The goodness of fit based on this analysis was good to adequate, with a 0.077 root mean square error of approximation.
CONCLUSIONS: Healthcare professionals and organizations are in a unique position to improve the health and well-being of their patients by implementing a trauma-informed approach to minimize the impact of adverse childhood experiences and trauma. This validated tool will allow organizations to identify gaps in knowledge, attitude, and practice among staff to subsequently begin developing pointed strategies to achieve a culture of trauma-informed practice.

INTRODUCTION

In the United States, two-thirds of individuals have been exposed to ≥1 traumatic event in their childhood. These exposures to trauma can place children at risk for emotional, physical, and functional impairment.[1] Although there are many definitions of trauma, the most commonly referred to definition is from the Substance Abuse and Mental Health Services Administration. They define trauma as “an event, series of events, or set of circumstances experienced by an individual as physically or emotionally harmful or life-threatening with lasting adverse effects on the individuals functioning and mental, physical, social, emotional, or spiritual well being.”[2] Although this definition provides context for healthcare professionals, it is important to note that trauma is a subjective experience; how an individual responds to trauma may evolve and change over time and well into adulthood.[3] The seminal study conducted by Felitti et al[4] underscores the impact of 10 specific childhood traumas, now commonly known as “adverse childhood experiences” (ACEs), by demonstrating how exposure to ACEs can lead to risky health behaviors and poor health outcomes in a dose-dependent manner. These findings, together with current research regarding the negative impact of childhood experiences on the future health and functioning of pediatric patients, serve as a call to action for healthcare professionals to incorporate trauma-informed care (TIC) into their routine clinical practice.[4-7]

TIC is an approach to healthcare that is rooted in the realization of the widespread incidence of trauma, recognition of the signs and symptoms of trauma, response to trauma in a full and integrated manner, and resisting retraumatization by organizations and healthcare professionals.[2] Furthermore, TIC includes adherence to the following key principles: (1) safety, (2) trustworthiness, (3) peer support, (4) collaboration, (5) empowerment and choice, and (6) cultural, historical, and sex issues.[2,8] TIC has historically been used among health professionals in social work, mental health, child services, and the juvenile justice program to inform their work, but it has not been routinely practiced by pediatric healthcare professionals.[1,8] Now with increasing evidence that TIC can lead to improved patient care outcomes, healthcare institutions are recognizing the importance of clinician competency in TIC.[9,10] In 2014, the AAP’s (American Academy of Pediatrics) “Trauma Toolbox for Primary Care” highlighted the role of a trauma-informed approach in addressing childhood adversity and trauma in the pediatric primary care setting.[11] Subsequent literature has highlighted the need for pediatric healthcare organizations to advance TIC[12] by educating healthcare professionals and developing competency in TIC.[13]

Regional data from the 2016 National Child Health Survey showed that 47% of the children in our region have had ≥1 ACE.[14] These data underscored the need for a coordinated approach to address trauma in the pediatric patients we care for at Children’s National Health System. As a first step, we assessed the learning needs of our target audience, a crucial first step in developing a learning curriculum.[15] Specifically, we wanted to measure knowledge, attitude, and practice (KAP) as they related to TIC to inform our educational intervention. The KAP method is a reliable and valid method to enhance KAP around a specific theme, establish baseline data, and suggest learner-centric intervention strategies.[16] For larger organizations, the best method of conducting a formal learning needs gap analysis is to use a valid and reliable survey tool.[15] Therefore, we searched for a validated tool for our needs assessment.

From our literature review, we found one study that measured nursing knowledge and experience related to TIC, but it was a qualitative study.[17] Another study by Bruce et al[18] surveyed providers working at a trauma center regarding TIC; however, this survey included questions related to posttraumatic stress, and these questions were not aligned with the intent of our work. Baker et al[19] was the only article that developed and validated a tool to measure TIC, the ARTIC (Attitudes Related to Trauma-Informed Care Scale). In this study, psychometric analysis established strong internal consistency and test–retest reliability for 45-item form with 7 subscales and a 35-item composite and 10-item short form. Although this tool was intended for use across disciplines, participants in the original study worked in education and human service settings.

Additionally, survey items were geared to measure the extent to which an individual or system is trauma-informed, with a lack of focus on understanding the important link between traumatic exposures and health outcomes. Missing from the available literature was a validated tool that measured KAP related to TIC among healthcare professionals within a healthcare setting. As such, this study aimed to validate a tool to assess KAP of TIC among interdisciplinary pediatric healthcare staff.

METHODS

Adopting Existing Survey Instrument

After a comprehensive literature review, we identified a survey tool that we could adapt for widespread use among healthcare professionals. The “Knowledge, Attitudes, and Practices of Trauma-Informed Practice Survey” is a 36-item unpublished and unvalidated tool developed by Abdoh et al.[20] Abdoh et al[20] designed the tool based on a literature review, then conducted face validity assessment with staff at their community center and pilot-tested the tool with 17 multidisciplinary staff members at that site. The community center provides housing to individuals with substance dependencies and concurrent physical and mental health issues. Its staff consist of various professional disciplines, including physicians, social workers, nurses, and mental health support staff. Among the 36 survey items, 29 questions measure knowledge, attitudes, and practice; 12 items related to knowledge, 11 related to attitudes, and 6 related to practices. Each item has a 5-point Likert scale response format with options from strongly disagree to strongly agree. The other 7 items of the survey capture demographic data, including the respondent’s role, department, years of experience, age group, and previous experience with TIC. We adopted this survey for our learning needs assessment after discussion with, and permission from, the author.

Face and Content Validity

To establish face validity, internal experts in the field of childhood adversity and trauma reviewed each item to assess the degree to which it would measure each of the 3 factors: KAP related to TIC. Internal expert qualifications included serving as faculty facilitators for continuing medical education on TIC institutionally, peer-reviewed publications related to mental health and trauma, employing TIC principles in their pediatric practice, coordinating federal and private funding on childhood trauma, and presenting at national conferences. We provided the following definitions of knowledge, attitudes, and practice to the internal experts as reference. Knowledge was defined as the degree to which staff members were aware of, informed about, or familiar with the applicability of TIC in the healthcare setting. Attitude was defined as how staff felt or thought about TIC, and practice was defined as how staff applied their knowledge and attitudes about TIC into their practice. We also asked these experts to review and assess whether the items on the tool should be included. Based on the review by the internal experts, only 1 item in the original tool, “Trauma-informed practice (TIP) shares many similarities to harm reduction,” was removed from the survey because our survey was being implemented in a pediatric healthcare institution whose staff is not well versed in harm reduction techniques. Additionally, the word “client” was replaced with “patient” to better align with the context of a large healthcare institution, versus a community setting. When determining content validity, institutional experts considered the item’s (1) relevance to TIP in general, (2) relevance to its associated factor, (3) alignment with common wording used at the institution, and (4) clarity of wording.

Data Collection

Before data collection, the organizational Institutional Review Board reviewed and approved the study. We collected the initial data between November and December 2017. An online survey platform, SurveyMonkey® (San Mateo, Calif.), was used to administer the survey tool, and the initial request to participate was sent via hospital email, followed by 2 subsequent email reminders until the survey closing date.

Construct Validity: Confirmatory Factor Analysis

The survey tool was adopted specifically for its ability to assess the 3 latent factors: knowledge, attitudes, and practices. Therefore, the priority for tool validation was to investigate whether the survey items are indeed manifest variables of the 3 factors.

Using confirmatory factor analysis (CFA) to assess the association between survey items and the 3 theorized latent factors, we assigned numeric values to each response for the analysis: Strongly disagree was assigned the value of 0, disagree a value of 1, neutral a value of 2, agree a value of 3, and strongly agree a value of 4. We assessed model fit based on factor loadings for individual survey items and goodness of fit tests commonly used in literature,[21-23] including chi-square test, the Bentler comparative fit index[24,25] (CFI), Tucker–Lewis index (TLI), and root mean square error of approximation (RMSEA). Akaike information criterion and Bayesian information criterion were also used to compare models. We tested survey items with a high residual for association with different factors based on modification indices.[26] Items sharing a standard covariance residual of ≥1.96 would prompt a review of the modification indices. If the item’s modification index was >3.84, it would then prompt an attempt to assess whether the item fits better with a different factor.[27] We removed items if the model fit did not improve as a result of changing the associated factor.[26,27]

Reliability: Internal Consistency

We assessed internal consistency by calculating Cronbach’s alpha for each factor and the overall tool each time CFA produced an acceptable model.[28] Using a combination of CFA and Cronbach’s alpha for survey tool validation is consistent with similar research in the field.[23,28]

RESULTS

Survey Respondents: Descriptive Data

We distributed the survey to 2,659 staff members. A total of 592 survey responses were collected—representing a 22.3% response rate. Among the responses collected, 37 were removed for no response on scale items, and 4 were removed for repeat entries. We deleted entries with missing scale item data listwise. Among the remaining 511 respondents, 54.4% were nurses, and 19.2% were attending physicians. Age distribution among the respondents was mostly even. A quarter of respondents (24.1%) had <2 years of experience in their current role, and 41.7% had ≥10 years in their current role. Approximately one-third of the respondents received some TIP training before completing the survey (Table 1).

Table 1.

Descriptive Data for the 511 Respondents Whose Questionnaires Were Used for Confirmatory Factor Analysis

Factor Analysis

We conducted CFA using Stata 15 software (StataCorp LLC, College Station, TX) to assess model fit and adjust the model to improve fit. The sample exceeds the commonly recommended minimum size of 300.[29] Data distribution for all items, except for one, meet multivariate normality. Therefore, we use maximum likelihood, which allows for mild departure for multivariate normality, as the estimation method for CFA.[21] As part of the model fitting process, we removed 7 items from the survey. The final 21-item model showed adequate overall model fit with a 0.077 RMSEA.[28] The chi-square statistics is 748.05 with a P value of <0.001. However, for sample sizes >500, it is common for the chi-square test to find statistical significance.[29] The CFI is 0.857, and the TLI is 0.839, each falls slightly below the 0.9 cutoff for “good fit.”[23-25] Altogether, the indices suggest an acceptable model. Internal consistency was good: Cronbach’s alpha values were within acceptable range with values of ≥0.74 for each of the 3 factors.[30] Specifically, the values were 0.84 for knowledge (with an item-to-total correlation range of 0.42–0.59), 0.74 for attitude (0.37–0.66), and 0.78 for practice (0.30–0.59).

The standardized interfactor correlation coefficients were 0.55 for knowledge–attitude (P < 0.001), 0.28 for knowledge–practice (P < 0.001), and 0.65 for attitude–practice (P < 0.001). The interfactor correlations suggest, by theory, some level of association among KAP factors for TIP. The medium to low value of the interfactor correlations also suggests that it is not redundant for the tool to include all 3 factors.

We reduced the total number of items from 28 to 21 in the final model (Table 2). In the final 21-item model, 7 items measure knowledge (5 items removed), 8 items measure attitude (2 items removed), and 6 items measure practice (0 items removed). The 7 removed items and any subset of the 7 do not form a fourth factor that would be theoretically meaningful.

Table 2.

Final 21-item Scale to Assess Staff Knowledge, Attitude, and Practice Related to Trauma-informed Care, Following Confirmatory Factor Analysis

DISCUSSION

This study aimed to establish the validity of the “Knowledge, Attitude, and Practice Related to Trauma-Informed Practice” tool. Our analysis indicated that the 21-item version could reliably assess KAP related to TIC among healthcare professionals in a pediatric institution. With increased awareness among healthcare professionals regarding the health impact of ACEs and trauma, healthcare organizations will need to effectively assess the learning needs of their staff to address gaps in KAP to implement a TIC approach that meets the needs of their patients.

At our institution, we used our survey results to develop and deliver an educational intervention for hospital staff. We partnered with the National Center on Trauma-Informed Care and Alternatives to Restraint and Seclusion, part of the Substance Abuse and Mental Health Services Administration, to provide 7 half-day learning sessions between January and March 2018. We have trained over 700 staff members and measured pre and post KAP using the validated KAP related to TIC tool. Our next steps include analyzing postintervention data and performing additional tool analysis that we will share in a future article.

Limitations

The first limitation of this work is that we adopted a preexisting tool that had not been previously assessed for validity and reliability. We based the initial item generation process on literature review findings, and we conducted the pilot test with a small sample—without assessment for construct validity. Recognition of these limitations was the primary motivation to validate the tool further using quantitative methods.

The second limitation is that we did not apply other forms of reliability and validity, such as alternate forms reliability, test–retest reliability, and criterion-related validity. We distributed a single version of the survey to hospital staff only once before we conducted the education program. Without distributing an alternate survey that worded questions differently, we could not assess alternate forms reliability. With a single round of baseline data, we could not examine the tool’s temporal stability. We conducted a literature search but could not find an existing “gold standard” scale for measuring TIC KAP for healthcare workers or workers in other professional sectors. Without a “gold standard” scale to compare against, we could not assess criterion-related validity.[31]

A third limitation is the generalizability of the survey. The primary objective of this study is to test the theorized item-factor relationships using CFA, for which the sample size is large enough. However, the sample is drawn exclusively from a single institution, and the response rate (22.3%) is low compared with typical online surveys.[32] Although there was diversity in the role and department affiliation, the possibility of bias exists. To make the scale truly generalizable for the field, we recommend that additional data be collected through a multicenter study approach from other healthcare agencies and institutions for further validation. Also, this study was done only in a pediatric institution. The original literature on ACE exposure was done in an adult cohort,[3] and therefore, it is well known that this science is relevant to adult populations as well. Further testing of this tool for applicability in adult healthcare institutions is needed because this work is of equal importance in the adult healthcare field.

Finally, goodness of fit indices CFI and TLI came close but still fell short of the commonly accepted threshold for good model fit. However, experts have recommended a balanced approach to consider both the theory and model fit. Too much focus on adhering to cutoffs of goodness of fit indices could lead to type 1 error, incorrectly rejecting an acceptable model.[23,33] In other words, theory and the RMSEA suggest that this is an acceptable model. It would be incorrect to reject it solely due to CFI and TLI values missing the commonly accepted thresholds. Having considered both the theory and the model fit, we believe our scale will be useful for implementing TIC in healthcare settings despite not meeting all of the goodness of fit indices.

CONCLUSIONS

We believe that healthcare organizations can take a proactive approach to improve the health and well-being of their patients by implementing a trauma-informed approach to minimize the impact of childhood trauma and adversity. Accurately assessing the needs of learners is an essential step in transforming the KAP among healthcare organizations and healthcare professionals as they relate to TIC. This tool will allow pediatric organizations to identify gaps in KAP among staff to subsequently develop strategies to achieve a culture of TIP.

DISCLOSURE

The authors have no financial interest to declare in relation to the content of this article.