Construct validity and reliability of the Tachikawa Resilience Scale in the Japan Ground Self-Defense Force personnel.

PURPOSE: The importance of resilience as a mental health outcome has been reported in recent occupational health studies, although resilience is yet to be assessed in the Japan Ground Self-Defense Force (JGSDF) population. Our objective was to test whether the Tachikawa Resilience Scale (TRS), developed to measure the resilience of Japanese individuals, is useful for evaluating the resilience of the JGSDF. PATIENTS AND METHODS: We performed a cross-sectional study of 353 JGSDF peacekeeping personnel engaged in the United Nations Mission in the Republic of South Sudan from November 2015 to May 2016. We evaluated resilience using two psychological measures: the TRS and the Resilience Competence Scale - Japanese Short Version (RCS-JS). To verify the construct validity of the TRS, we performed exploratory factor analysis and confirmatory factor analysis. We subsequently conducted hierarchical multivariate regression analysis to evaluate the relationship of the TRS and the RCS-JS with psychological distress measured by the Japanese version of the Kessler Psychological Distress Scale.
RESULTS: Of those recruited, 281 (79.6%) agreed to participate. The exploratory factor analysis revealed a one-factor model of the TRS. The confirmatory factor analysis model showed good fit (ratio of χ 2 to the degrees of freedom =1.409, P=0.105, comparative fit index =0.994, root mean square error of approximation =0.038). Both the TRS and the RCS-JS showed a significant inverse correlation with the Kessler Psychological Distress Scale, and the regression coefficient of the TRS was equivalent to that of the RCS-JS.
CONCLUSION: We confirmed the construct validity and reliability of the TRS when applied to the JGSDF, and demonstrated the usefulness of the TRS in this population.

Introduction

Experience of extremely stressful events can affect people’s mental health and cause depression, posttraumatic stress disorder, or other psychiatric disorders.1 Resilience, which is defined as the ability to maintain a stable equilibrium, has been proposed as a recovery factor against stress.2 The construct of resilience and its factors are complex and may be specific for a particular individual, family, organization, society, and culture.1 The importance of resilience has been addressed in the domains of occupational health and military health – military personnel may experience severe events during their deployment, including combat and other life-threatening situations, as well as witnessing atrocities.3

The best way to evaluate resilience has yet to be determined. Resilience rating scales widely used around the world were typically created in Europe and the United States, such as the Resilience Scale (RS) and the Connor–Davidson Resilience Scale.4,5 Although there is a Japanese version of the RS,6 it has not exhibited the same utility as the original because the validity of this Japanese version has been considered low compared to that of the original.6 One reason for this is a difference in the cultural sensitivity to resilience between Japan and Western countries. Accordingly, the Tachikawa Resilience Scale (TRS) was developed to measure the resilience of Japanese populations.7 However, to date, the subjects of TRS reports have been limited to company workers and patients with psychiatric disorders.7

In the context of military mental health, the Resilience Competence Scale (RCS) has been used to verify the effect of resilience training on US Army members.3 Although the validity of the RCS – Japanese Short Version (RCS-JS) has been established in the Japan Ground Self-Defense Forces (JGSDF),8 the original version was developed in the United States and does not reflect the Japanese cultural background.

The JGSDF has been engaged in a large number of United Nations peacekeeping operations and disaster relief activities. Through these JGSDF activities, the personnel are potentially exposed to traumatic events, and it is important to gather knowledge about the resilience of personnel. However, to the best of our knowledge, there are no established standard scales by which to measure resilience in the JGSDF.

The primary endpoint of this study was to examine the factor structure of the TRS with exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). The secondary endpoint was to examine the construct validity of the TRS and compare it to that of the RCS-JS and the Japanese version of the Kessler Screening Scale for Psychological Distress (K6)9 using a correlation analysis and a hierarchical multivariate regression analysis.

Materials and methods

Participants

Participants were all JGSDF personnel who were deployed to a peacekeeping mission – the United Nations Mission in the Republic of South Sudan – from November 2015 to May 2016 (n=353). Unlike other countries’ troops, a very large majority of the JGSDF personnel is Japanese. In addition, the country has not experienced traumatic events because of its constitutional constraints, and the JGSDF activities are limited to noncombat areas. The United Nations Mission in the Republic of South Sudan was tasked to consolidate peace and security and to help establish conditions for the development of South Sudan.

The exclusion criteria for this study consisted of refusal to receive analysis. Of the 353 JGSDF personnel, 281 individuals (79.6%) agreed to participate in this study. The self-reported questionnaire was administered in January 2016 – during their mission engagement.

We collected the following demographic information: age, sex, military rank (officer, enlisted/private), marital status, whether the individual had children, and previous deployment experience. In previous studies, older age, the existence of a spouse or child, and previous deployment experience corresponded to good mental health,10,11 while being female or of a lower rank did not.12–15

Measures

We used two psychological measures to evaluate resilience: the TRS and the RCS-JS. The TRS is a 10-item self-administered questionnaire used to evaluate resilience. Response are obtained via a 7-point scale ranging from 1 (strongly disagree) to 7 (strongly agree); the total score ranges from 10 to 70.7 Higher scores reflect higher resilience. The original text of the TRS is in the Japanese language, and several items reflect Japanese culture-bound cognitions. For example, items such as “I accept things as they are when there are no alternatives” and “I try not to worry about what is beyond my capabilities” can be regarded as culturally appropriate for Japanese individuals because these items reflect the idea of Morita therapy, which guides patients to accept anxiety as it is.7

The RCS-JS is a 12-item self-administered questionnaire used to evaluate resilience. Responses are obtained via a 5-point scale ranging from 1 (disagree) to 5 (agree), with a total score ranging from 12 to 60.8 The original RCS is a 20-item self-administered questionnaire with a total score ranging from 20 to 100.3 Higher scores reflect higher resilience. The validity of the RCS was reported in a study that examined the efficacy of the US Army’s resilience training program among US Army National Guard soldiers. The program addressed six core competencies of resilience: connection, optimism, mental agility, self-awareness, self-regulation, and character strength. The RCS was created to assess these six competencies.3 The study showed that the developed measure of Master Resilience Training (MRT) core competencies had adequate reliability and validity for purposes of examining the intended effects of the MRT training.3

We evaluated the participants’ degree of psychological distress using the Japanese version of the K6.9 The K6 is a 6-item self-administered questionnaire that is widely used to screen for mood and anxiety disorders according to the definition in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV).13,16 K6 responses are collected via a 5-point scale ranging from 0 (never) to 4 (always), with a total score ranging from 0 to 24. Higher scores reflect higher psychological distress.

Statistical analyses

A Mann–Whitney U-test was used to compare the averages of continuous data on demographic characteristics. EFA was used to determine the factor structure of the TRS. A principal components analysis, Kaiser’s criterion, and a scree test were used to determine the number of factors. We subsequently performed CFA to verify the factor structure of the TRS using structural equation modeling. The model fit of the CFA was evaluated using the following indicators: the ratio of χ2 to the degrees of freedom (CMIN/df), the goodness of fit index (GFI), the comparative fit index (CFI), and the root mean square error of approximation (RMSEA). The acceptable threshold levels for these indicators were as follows17: CMIN/df <3,18 GFI and CFI >0.95,17 and RMSEA <0.07.19

Spearman correlations were calculated to evaluate the relationship between the TRS and other theoretically related measures, namely the RCS-JS and the K6. A Z-test was conducted to evaluate the difference between the TRS and the RCS-JS correlation coefficients. We subsequently conducted hierarchical multivariate regression analysis to evaluate the relationship between the TRS and the RCS-JS with the K6 (dependent variable). In Step 1, age, sex, military rank (officer, enlisted/private), marital status, having children, and previous deployment experience were entered as covariates. In Step 2A, the TRS was entered as a covariate, and in Step 2B the RCS-JS was entered likewise.

We assessed the reliability of the TRS by examining its internal consistency using Cronbach’s α coefficient. All statistical analyses were conducted using IBM SPSS Statistics 24 (IBM Corporation, Armonk, NY, USA) and IBM Amos 24 (IBM Corporation).

Results

Table 1 shows the demographic characteristics of the deployed personnel and scale scores of the TRS, RCS-JS, and K6. The total score of the TRS was significantly higher for males (median, IQR; 53.0, 46.5–59.5) than for females (46.5, 43.3–53.0) (P=0.048). The total score of the RCS-JS was significantly higher for personnel who had previous deployment experience (median, IQR; 47.0, 43.0–55.0) than for those who did not (46.0, 41.0–51.0) (P=0.035).

Table 1

Demographic characteristics of deployed personnel and scores of the TRS and other measures

Demographic characteristics	n (%)	Median (IQR)
Total	281 (100)
Age		33.0 (28.0–38.0)
Males	269 (95.7)
Military rank, officer	61 (21.7)
Married	163 (58.0)
With children	140 (49.8)
Previous deployment experience, yes	61 (21.7)
Measures scores
TRS		52.0 (46.0–59.0)
RCS-JS		46.0 (69.0–85.5)
K6		1.0 (0.0–3.0)

Abbreviations: IQR, interquartile range; K6, The Kessler Screening Scale for Psychological Distress; RCS-JS, The Resilience Competence Scale – Japanese Short Version; TRS, The Tachikawa Resilience Scale.

Construct validity

The EFA revealed only one factor with an eigenvalue (5.34) >1. This factor included 53.4% of the variance. The scree plot showed that one factor was suitable (Figure 1). The factor loadings were between 0.58 and 0.78 for the ten resilience items.

Figure 1

Scree plot of the TRS.

Notes: EFA revealed only one factor with an eigenvalue (5.34) >1. This factor included 53.4% of the variance. This scree plot shows that one factor was suitable.

Abbreviations: EFA, exploratory factor analysis; TRS, The Tachikawa Resilience Scale.

The CFA model showed good fit properties (CMIN =28.178, df=20, CMIN/df=1.409, P=0.105, GFI =0.981, CFI =0.994, RMSEA =0.038, and AIC =98.178). The individual item loadings for the factors are shown in Table 2.

Table 2

Means, standard deviations, correlations with the TRS total score, Cronbach’s α if item was deleted, and factor loading of each item of the TRS

TRS item	Mean (SD)	TRS total score		Cronbach’s α if item was deleted	Factor loading
		ρ	P-value
1	5.7 (1.1)	0.77	<0.001	0.89	0.61
2	5.6 (1.1)	0.70	<0.001	0.89	0.58
3	5.1 (1.5)	0.78	<0.001	0.89	0.78
4	5.1 (1.4)	0.75	<0.001	0.89	0.76
5	5.2 (1.4)	0.65	<0.001	0.90	0.60
6	5.2 (1.3)	0.76	<0.001	0.89	0.76
7	5.1 (1.5)	0.72	<0.001	0.89	0.69
8	4.9 (1.5)	0.76	<0.001	0.89	0.68
9	5.5 (1.2)	0.71	<0.001	0.89	0.66
10	5.6 (1.3)	0.64	<0.001	0.90	0.59

Note: ρ = Spearman’s rank correlation coefficient.

Abbreviation: TRS, The Tachikawa Resilience Scale.

The total score of the TRS was significantly associated with that of the K6 (r=−0.499, P<0.001). The total score of the RCS-JS was also significantly associated with that of the K6 (r=−0.484, P<0.001). There was no significant difference between these two correlation coefficients (z=0.233, P=0.816). The total score of the TRS was significantly associated with that of the RCS-JS and its six domains (Table 3). The multivariate regression analysis revealed that the TRS and the RCS-JS were negatively related to the K6. The regression coefficient of the TRS was not lower than that of the RCS-JS (Table 4).

Table 3

Correlation between the TRS and the RCS-JS

Variable	RCS-JS
	Total	Connection	Optimism	Mental agility	Self- awareness	Self- regulation	Character strength
TRS total
ρ	0.64	0.34	0.66	0.54	0.41	0.59	0.48
P-value	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001

Note: ρ = Spearman’s rank correlation coefficient.

Abbreviations: RCS-JS, The Resilience Competence Scale – Japanese Short Version; TRS, The Tachikawa Resilience Scale.

Table 4

Results of hierarchical multivariate regression analysis

Variable	K6 (n=281)
	R2	ΔR2	P-value	B	SE	β	P-value
Step 1	0.07	0.07	0.002
Age				0.08	0.03	0.22	0.005
Sexa				1.74	0.81	0.13	0.033
Military rankb				−0.61	0.43	−0.12	0.062
Marital statusc				−0.54	0.55	−0.10	0.327
With childrend				−0.22	0.54	−0.04	0.679
Previous deployment experiencee				−0.33	0.41	−0.05	0.424
Step 2A	0.25	0.18	<0.001
Age				0.05	0.03	0.13	0.064
Sexa				0.95	0.74	0.07	0.199
Military rankb				−0.69	0.29	−0.14	0.019
Marital statusc				−0.20	0.50	−0.04	0.690
With childrend				−0.52	0.49	−0.09	0.294
Previous deployment experiencee				−0.01	0.37	−0.00	0.982
TRS				−0.12	0.02	−0.43	<0.001
Step 2B	0.23	0.16	<0.001
Age				0.06	0.03	0.15	0.030
Sexa				1.38	0.74	0.10	0.065
Military rankb				−0.58	0.29	−0.12	0.051
Marital statusc				−0.51	0.50	−0.09	0.315
With childrend				−0.24	0.50	−0.04	0.622
Previous deployment experiencee				0.12	0.38	0.02	0.747
RCS-JS				−0.14	0.02	−0.40	<0.001

Notes:

0= male, 1= female.

0= officer, 1= enlisted/private.

0= unmarried, 1= married.

0= no, 1= yes.

0= no, 1= yes.

Abbreviations: K6, The Kessler Screening Scale for Psychological Distress; RCS-JS, The Resilience Competence Scale – Japanese Short Version; TRS, The Tachikawa Resilience Scale.

Reliability

The Cronbach’s α coefficient of the 10-item TRS was 0.90, showing very good internal consistency. Elimination of any question item(s) did not significantly increase this value (Table 2).

Discussion

According to the present study, the construct validity and reliability of the TRS were acceptable in a sample of JGSDF personnel. The results supported a one-factor model of the TRS and showed good internal consistency. The TRS was inversely correlated with the K6. To the best of our knowledge, this is the first study to show this relationship within the personnel of the JGSDF or other services of the Japan Self-Defense Forces.

The TRS regression coefficients were comparable to those of the RCS-JS, even though the TRS questions differ slightly from those of the RCS-JS, which consists of six domains (connection, optimism, mental agility, self-awareness, self-regulation, and character strength). Some items show similarities between the scales; for example, we can regard two TRS items (“Even during hardships, I think I will be able to manage” and “Even during hardships, I can maintain a normal state of mind”) as similar to optimism and mental agility items, respectively, of the RCS-JS. In contrast, no TRS item corresponds to connection in the RCS-JS. The RCS defines connection as an “ability to develop and maintain strong interpersonal relationships through effective communication, empathy toward others, willingness to ask for help, and supporting others.”3 Therefore, this item represents social support, which is an important component of resilience.20 We speculate that the link between the TRS and the Japanese cultural norms compensates for the lack of an item that corresponds to social support. However, future studies will be needed to verify this speculation.

Another characteristic of Japanese culture is a tendency to suppress the expression of positive affect.21 Therefore, Japanese individuals might tend to generate low scores on some RCS-JS items (eg, “Confident in handling stressful circumstances” or “Control my emotions”). In contrast, the TRS items that reflect the idea of Morita therapy7 might be answered naturally by Japanese individuals. TRS items nos 2 and 7 (eg, “I accept things as they are when there are no alternatives” and “I try not to worry about what is beyond my capabilities”) could be culturally sensitive, as mentioned above, and their factor loadings are 0.58 and 0.69 (Table 2). Contrary to expectations, these values were not very high, which could reflect the cultural implications of the scale. Future studies should assess these cross-cultural comparisons between the TRS and other resilience scales of Western origin. Convergent validity was shown by the significant correlation between the TRS and the RCS-JS (Table 3). The correlation coefficient between the TRS and connection in the RCS-JS was relatively low compared to the other five domains. This result was considered to be due to the abovementioned lack of the TRS item that corresponds to social support.

To date, the utility of the TRS has been reported with company workers and patients with psychiatric disorders.7 In this study, we showed the usefulness of the TRS for the JGSDF personnel. Therefore, we could use the knowledge about resilience obtained from populations other than the military by using the TRS. In addition, we could also apply the findings gained from the military to other populations.

Several limitations of this study should be acknowledged. First, the theoretical rationale for the TRS may not be strong, because it was developed on the basis of data obtained from unstructured interviews with motor vehicle accident survivors.7 Nonetheless, the TRS contains cultural aspects, which influence the way that resilience is constructed.1,7 Second, there is no item concerning social support in the TRS. Third, there was selection bias in this study because the participants were limited to dispatched members of the JGSDF. Fourth, we conducted the EFA and the CFA using the same sample group because of the sample size, which could have led to similar results between the measures. Fifth, there was lack of comparison to measures that evaluate resilience other than the RCS-JS. Finally, the study design did not allow us to determine any causal associations between resilience and mental health status.

Conclusion

We established the construct validity and reliability of the TRS when applied to the JGSDF. Therefore, the TRS could be a useful resilience evaluation scale for the JGSDF. In the future, a more valid and reliable resilience evaluation scale could be created for JGSDF personnel by adding items related to social support to the TRS.

Ethics statement

This research followed the principles outlined in the Declaration of Helsinki. This study was conducted with the approval of the Ethics Committee of the National Defense Medical College (Approval No 2404). All participants provided written informed consent.