Development and psychometric evaluation of the Treatment Adherence Questionnaire for Patients with Combat Post-traumatic Stress Disorder.

PURPOSE: Treatment adherence is one of the major strategies in treating post-traumatic stress disorder (PTSD) in combat veterans. This study developed and psychometrically assessed the Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder. PARTICIPANTS AND METHODS: This methodological study was conducted in Tehran, Iran, during 2016-2017 in two phases. First, the concept of treatment adherence in combatants with PTSD was analyzed using a hybrid model. This model consisted of three phases: literature review phase, fieldwork phase, and final analysis phase. The consequences and attributes of the concept of treatment adherence in combatants with PTSD were identified, and based on the findings, the Treatment Adherence Questionnaire for PTSD veterans was developed. In the second stage, the face and content validities of the questionnaire were investigated both quantitatively and qualitatively. Exploratory factor analysis and confirmatory factor analysis were used to determine the questionnaire's validity. Internal consistency correlation coefficient of the questionnaire was estimated with Cronbach's alpha coefficient, while the reliability of the questionnaire was established using intra-class test-retest correlation coefficient. Study participants were selected from inpatients and outpatients referred to a hospital, clinic, and health center in Tehran and Kashan, Iran. All patients were diagnosed with combat PTSD by a psychiatrist, based on psychiatric interview and other clinical findings.
RESULTS: The Persian version of the Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder included 17 items. Exploratory factor analysis identified three factors which accounted for a total of 87.57% of the total variance of treatment adherence score. The identified factors were labeled as "maintenance of treatment", "follow-up and treatment contribution", and "purposefulness and responsibility". The Cronbach's alpha correlation coefficient was 0.92 and the intra-class correlation coefficient of the questionnaire's reliability was estimated at 0.92 (P<0.001).
CONCLUSION: The data obtained confirmed the hypothesis of the factor structure model with a latent second-order variable. The final version of the Treatment Adherence Questionnaire for Iranian combatants with PTSD can be applied as a valid and reliable questionnaire for measuring treatment adherence in these patients.

Introduction

Post-traumatic stress disorder (PTSD) is considered as one of the most important effects of war on combatants.1 There are no accurate statistics on the prevalence of this condition among the combatants surviving the war; however, Zhen et al reported the prevalence rate of this disorder in their systematic review as 34% among the surviving combatants.2 Despite the passage of about 3 decades since the end of the Iraqi war imposed against Iran, there are no exact statistics on the prevalence of this calamity among the surviving Iranian combatants (the veterans who were in the Iran-Iraq war). Noorbala in their study reported a prevalence rate of 27%.3

The chronic and stressful nature of this disorder along with its associated complications and consequences may affect the various aspects of individual, familial, and social life of these patients in different ways, leading to reduced quality of life and diminished satisfaction with life in these patients and their families.2,4,5 The most important complications of this disorder include issues like the incidence of physiological reactions, nightmares, sleep disorders, interactive reactions like flashbacks, mood disorders, depression, accommodation disorders, decreased self-esteem, irritability, violence, quarrels, social isolation and seclusion, limited emotions, concentration problems, suicidality, and memory impairments.6,7

Fortunately, a wide range of pharmaceutical and non-pharmaceutical treatments (psychotherapy, behavioral therapy, and cognitive therapy) are available today for improving combat PTSD and its consequences.8,9 The refractory nature and its persistence against treatment, along with other social-individual factors, have led to the failure of many recommended therapeutic methods in these patients.8,10 Success of these treatment modalities is dependent on appropriate patient adherence to treatment in this patient population.8,11

According to WHO, treatment adherence is defined as the correspondence of one’s behavior (consumption of medicines, diet, and changes in lifestyle) on the basis of recommendations offered by health care providers.12 Many models and theories have attempted to explain adherence to treatment or non-adherence to treatments in people with mental disorders. Some of the most important models that have addressed this issue are Self-Regulation Model, Theory of Planned Behavior, and Health Belief Model.13

There is not much information available on treatment adherence in combat PTSD patients.12,14–17 The few studies conducted so far in this field have indicated that these patients manifest no proper treatment adherence.17–19 One probable cause of the limited number of such studies is the lack of a suitable specialized measurement questionnaire for assessing the rate of treatment adherence in these patients. There are various methods to measure treatment adherence, and they can be classified into six categories (biologic measurements, secondary database analysis, technology-assisted monitoring measures, pill count, clinician assessment of treatment adherence, questionnaires, and scales) (Table 1).18–21 Among the six methods of treatment adherence measurement, the following two methods, pill counts and biologic indices measurement, are not applicable in Iran due to high costs to patients and the health care system, and insufficient equipment and facilities accessible to all surviving combatants with PTSD. Some limitations exist in the use of secondary data and the use of technological and electronic systems in treatment adherence measurement and include lack of a unified national network of health system, weak infrastructures for the use of modern technologies and equipment in Iran and many developing countries. The use of interviews and self-report questionnaires may be highly useful in exploring treatment adherence in these patients in countries like Iran. Nonetheless, these methods suffer from deficiencies such as time-consuming interviews, the technical process of interviewing, interviewer’s ability to conduct an interview, non-specified questionnaire for combatants with PTSD, complexity and difficulty of scoring the questionnaires, and unidirectionality of the instruments.18–21

Table 1

Methods for treatment adherence measurement

Method	Example
Biologic measurements	Measurements of a drug or its metabolite in serum, urine, saliva
Secondary database analysis	Medication possession ratio, dichotomous variable, continuous, multiple interval measure of medication acquisition
Technology-assisted monitoring measures	Electronic medication packaging devices, medication event monitoring or MEMS caps, Med-eMonitor, eCaps, and Medsignals
Pill count	Not applicable
Clinician assessment of treatment adherence	Patient-kept diaries, patient interviews
Questionnaires and scales	Brief Medication Questionnaire, Hill-Bone Compliance Scale (Hill-Bone), Medication Adherence Questionnaire, and Medication Adherence Report Scale

Considering the importance of treatment adherence in the course of treatment of these patients, along with limitations and deficiencies of the instruments and methods presently available for treatment adherence as well as lack of appropriate specialized commonly acceptable instruments to be used in measuring treatment adherence in combat PTSD, the aims of this study were development and psychometric validation of treatment adherence measurement questionnaire in PTSD veterans.

Material and methods

Design and setting

This methodological study was conducted in two phases in Tehran, capital of Iran, in 2016–2017.

Characteristics of participants

The convenience sampling method was applied; accordingly, a purposeful sample of 269 known combat PTSD cases were recruited. Study participants were selected from inpatients and outpatients who were referred to Baqiyatallah Hospital, Clinic of Baqiyatallah Hospital, and Ayatollah Sadr Psychiatric Hospital in Tehran, and the Foundation of Martyrs and Veterans Affairs in Kashan, Iran.

All samples were diagnosed with combat PTSD by a psychiatrist, based on psychiatric interview and other clinical findings. The inclusion criteria were Iranian nationality, ability to understand and speak Persian, ability to respond to the questionnaire items, having no history of known psychosis or cognitive impairments, diagnosis of PTSD due to the presence in the Iran-Iraq war. Individuals who chose to withdraw from the study or were reluctant to answer the questionnaire items were excluded from the study.

Data collection questionnaires included a demographic questionnaire (with items on age, age of the participant at the time of trauma, marital status, present occupation, social activities, history of mental disorders before the trauma, cause of trauma, and economical status) and the Treatment Adherence Questionnaire.

Description of all processes

Item generation

In the primary phase, a qualitative study was performed using a concept analysis method and hybrid model approach (literature review and field work). In the literature review, section 11 documents were reviewed, and in the fieldwork, 12 participants (four patients with combat PTSD, four family members of the patient, and four treatment team members) were interviewed. All family members of patients were informed of the patient’s condition.

The participants were chosen purposefully. Individual, face-to-face, in-depth, and semi-structured interviews with the participants continued until data saturation. The period of interviews varied between 40 and 210 minutes (mean: 52 minutes). All interviews were conducted by an individual (corresponding author) in Iranian-Islamic context.

In sampling, the maximum diversity was considered for age, marital status, educational level, clinical experience, and treatment status. Sampling continued until the data were saturated.

The interview guiding questions at this phase included the following:

What factors will make the patient (you) better follow the instructions of the treatment team and the doctor?

What happens if the patient (you) decides to leave therapeutic regimen?

The interviews were implemented as soon as possible by the corresponding author. The data from literature review and fieldwork were managed and organized using MAXQDA software, version 10. To ensure the trustworthiness of qualitative data used, the Lincoln and Guba criteria, including credibility, confirmability, dependability, and transferability were considered.

The concept analysis method of Walker and Avant22 was used to analyze literature texts and for analysis of the interview texts Graneheim and Lundman23 content analysis method was used. The texts (from literature and interview) were read word for word, line by line, and paragraph by paragraph several times for better understanding. Next, some parts of the text were selected; and the meaning of units and initial codes were extracted. Then, similar codes were classified as sub-categories. The sub-category formed categories together. An attempt was made to create homogeneity within the category but the highest heterogeneity between the categories. The categories also formed the themes. Item pool was formed on the basis of the consequences and attributes of treatment adherence in combat PTSD.

Item reduction

First, 56 items were designed based on the findings obtained from the first phase of this study. Then, 26 items were selected as the first draft of the questionnaire by a panel of experts which included a psychiatric nurse, a psychiatrist, a psychologist, and two experts of qualitative studies.

Validation

Face and content validities

To evaluate face validity, ten patients were individually interviewed and their comments regarding the complexity, clarity, and the ambiguity of the items were used to modify the contents. To assess content validity, ten experts on combat PTSD (psychologist, psychiatrist, and psychiatric nurse) were used. The necessity of each item was evaluated two times using content validity ratio (CVR) in a 3-point Likert scale (1= necessary to 3= not necessary), and decision-making took place according to the critical values of the Lawshe table. In addition, the relevance of items was determined using CVI. By using a 4-point Likert scale for assessment of CVI, the following criteria were used: relevance, importance, clarity, and simplicity. Walts and Basel index was used to decide which CVI items to use. Ten items did not qualify for inclusion at this stage and were removed from the scale.

Construct validity assessment

Construct validity was evaluated using the maximum likelihood exploratory factor analysis (MLEFA) and confirmatory factor analysis (CFA). The Bartlett’s test of sphericity and Kaiser-Meyer-Olkin (KMO) index were used to evaluate sampling adequacy; KMO >0.8 indicated an adequate sample.24 The factors were extracted by MLEFA using Promax rotation with scree plot. The presence of an item in an extracted factor was determined as approximately 0.3 using the equation critical value =5.152/√(n-2), where n = sample size.25 The factors extracted using the first- and second-order CFA and the most common goodness-of-fit indices of the recommended model were evaluated. Fit indices employed in the study included chi-squared (χ2), chi-squared/degrees of freedom ratio (normalized chi-squared CMIN/df), adjusted goodness-of-fit index >0.8, incremental fit index >0.90, parsimonious comparative fit index and parsimony normed fit index >0.50, comparative fit index >0.90, and root mean square error of approximation <0.05.25

In the second-order CFA, it was assumed that the extracted latent factors in the first stage were present. Thus, the second-order CFA represented the more general concepts at secondary and upper levels.26

Convergent and divergent validity

The convergent and divergent validity of the Treatment Adherence Questionnaire for Patients with Combat Post-traumatic Stress Disorder was assessed using the average variance extracted (AVE), the maximum shared squared variance (MSV), and the average shared squared variance (ASV). The convergent validity is established when AVE >0.5 and divergent validity is established when both MSV < AVE and ASV < AVE.27

Reliability

To assess the internal consistency of the scale, coefficients of Cronbach’s alpha, McDonald Omega, and Theta were estimated and values higher than 0.7 were considered acceptable.28 The construct reliability (CR), which replaces Cronbach’s alpha coefficient in structural equation modeling, was then assessed, and CR greater than 0.7 was considered acceptable.29 Stability of the questionnaire was determined using test-retest reliability method on 50 samples selected randomly from among the total number of samples under study during a two week interval and investigated by estimating the Pearson product moment correlation coefficient and intra-class correlation (ICC).

Ethical considerations

Approval of code of ethics in research was obtained on the basis of the Helsinki Treaty from the Committee of Ethics in Research at Baqiyatallah University of Medical Sciences, moreover, the participants and their families were clearly informed about research goals and objectives and were assured that they could voluntarily exit the study at any stage. Patient information was kept confidential. Also, written informed consent was obtained from the patients or their families. The required data were gleaned by a PhD student of gerontology (corresponding author).

Statistical analysis

SPSS-AMOS24 were used for data analysis. Multivariate normality was assessed by Mardia coefficient of multivariate kurtosis. Moreover, the rate of standard error of measurement (SEM = SD × √1-ICC) and smallest detectable change (SDC =1.96 × √2 × SEM) were calculated for the questionnaire.

Results

Findings of the first stage

A total of 117 codes extracted in this phase (47 codes from review of literature and 70 codes from interviews) were categorized into a theme of consequences (changes in lifestyle) and another theme of characteristics (planning for health promotion).

Improved quality of life due to treatment adherence was identified as the most fundamental consequence of treatment adherence in combatants with PTSD. This major theme consists of the two main categories of disease recurrence and progressive disorder in role playing. The category of disease recurrence consists of three sub-categories of mental manifestation due to lack of treatment adherence, disease deterioration, and treatment consequences. Along with aggravation of disease signs and symptoms, inaccessibility of suitable therapeutic outcomes, less than expected effectiveness of treatments, and persistence of symptoms have also led the patients not to show any inclination for continuing the treatment. The other sub-category in this phase was progressive disorder in role playing. The effects of lack of treatment adherence can be observed in the inability of patients in playing the individual, familial, and socioeconomic roles. Following the lack of adherence to treatment, the patients will experience more severe symptoms and higher levels of mental manifestation such as stress, depression, anxiety, irritability, and violence resulting in increased problems in patient’s engagement with family members, finally leading to the loss of ability to play family roles.

The most important theme found in attributes of patients with treatment adherence was planning for health promotion programs. This is done through responsibility for treatment. Patients with appropriate treatment adherence follow-up their treatment because they enjoy the positive attitude toward treatment, and have good therapeutic rapport with the treatment team. Additionally, responsibility for treatment drives the combat PTSD patients to adapt themselves to their disease and plan for their discharge program as they accept their disorder and makes them “compatible” with it, to have a prospective purposeful life.30

Findings of the second phase of the study: validity and reliability measurement

All of the participants were male. About 87.5% of the participants were married, 36.8% were unemployed, and 79.2% had no social activities. The means of participants’ age and age at the time of trauma scores were 55.57±5.19 and 24.50±5.28, respectively (Table 2).

Table 2

Demographics and contextual information of the assessed samples

Variable	N (%)
Marital status
Married	235 (87.5)
Single	12 (4.5)
Divorced	19 (7.1)
Marital discontinuation	3 (1.1)
Present occupation
Unemployed	99 (36.8)
Self-employment	81 (30.1)
Employed	7 (2.65)
Military employment	0 (0)
Retired	82 (30.5)
Social activities
Yes	31 (20.8)
No	217 (79.2)
History of mental disorders before trauma
Yes	246 (91.4)
No	23 (8.6)
Trauma type
Explosion wave	131 (48.7)
Observation of scenes of martyrdom and injuries	55 (20.4)
Engagement in a disorganized situation	27 (10.10)
Captivity	30 (11.2)
Amputation	26 (9.7)
Mean ± SD
Age of the participants (years)	55.57±5.19
Age of the participants at the time of trauma (years)	24.50±5.28
Economical status (visual analog scale 0–10)	5.29±1.42
Percentage of disability	41.10±12.30
Cases of hospitalization per year	2.5±0.94

Validity

Face and content validity

Upon qualitative measuring of the face validity from the viewpoint of experts and the target group, a few items were modified for better understanding. These changes were mostly applied to the wording and appearance of the items. Regarding quantitative face validity, the score of each individual item was greater than 1 and the CVR rate of the second phase for each item equaled 1. The context validity index for each item in rela­tion to the three criteria of relevance, simplicity, and clarity was estimated between 0.8 and 1. The content validity index of the whole questionnaire was (S-CVI Ave) 0.96 and the construct validity of the questionnaire was investigated using 17 items.

Construct validity

The sampling adequacy (KMO) was calculated as 0.926 and Bartlett’s test was calculated as χ2=2,218.48, df =136 (P<0.001). The exploratory factor analysis (EFA) resulted in the extraction of three factors (factor 1: maintenance of treatment [six item], factor 2: follow-up and treatment contribution [five item], and factor 3: purposefulness and responsibility [six item]) which explained 87.57% of the total variance (Table 3).

Table 3

Exploratory factor analysis of Persian version of the Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder

Factors	Items	Factor loading	Communalities	Eigenvalue	% of variance
1	Q12. In the event of any complication, I would contact the treatment team and would ask for advice	0.831	0.612	5.80	30.00
	Q10. I renew my prescription without visiting a doctor	0.706	0.504
	Q11. I buy all medications prescribed by the doctor	0.702	0.595
	Q13. I take my medications with me on my travels and parties or when leaving home	0.702	0.581
	Q14. I stop taking medications without consulting a doctor, whenever I feel that they are not helpful enough	0.571	0.541
	Q9. During the past month, whenever I have felt that my illness is under control, I have reduced/stopped my medications	0.514	0.260
2	Q17. During the past month, I have adjusted my daily schedule with my treatment schedule	0.855	0.675	5.62	29.57
	Q18. During the past month, adherence to the proper implementation of my treatment schedule has been important to me	0.728	0.559
	Q16. I’m fulfilling my responsibilities I have undertaken for my treatment very well	0.715	0.415
	Q15. I pursue a specific goal in each treatment course	0.689	0.578
	Q19. During the past month, I have had clear plans and goals to recover from my illness	0.555	0.444
3	Q3. During the past month, I have been following my non-pharmacological treatments (psychotherapy, etc)	0.968	0.771	5.43	28.00
	Q4. I visit my doctor on time	0.813	0.658
	Q2. During the past month, I have needed others to remind me of my treatment schedule (eg, taking medication, visiting a clinic or a laboratory)	0.560	0.382
	Q5. I only visit my own doctor and refrain from visiting other doctors	0.457	0.440
	Q7. I am actively involved in my treatment procedures and express my own views	0.368	0.322
	Q6. I attend classes relevant to my illness	0.357	0.426

Notes: Factor 1: maintenance of treatment [six item], factor 2: follow-up and treatment contribution [five item], and factor 3: purposefulness and responsibility [six item].

Abbreviation: Q, question.

In the first CFA, the results of the chi-squared test were obtained as χ2 (116)=274.49 (P<0.001). According to Table 4, all the indices (parsimonious comparative fit index =0.792, parsimony normed fit index =0.750, CMIN/df=2.366, root mean square error of approximation =0.071, adjusted goodness-of-fit index =0.891, incremental fit index =0.921 and comparative fit index =0.926) confirmed a good fit of the first final model (Figure 1).

Table 4

Fit indices of the first- and second-order confirmatory factor analysis of the Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder

CFA	χ2	df	P-value	CMIN/df	RMSEA	PCFI	PNFI	AGFI	IFI	CFI
First-order after structure modification	274.49	116	<0.001	2.366	0.071	0.792	0.750	0.850	0.921	0.926
Second-order after structure modification	193.02	113	<0.001	1.702	0.051	0.880	0.760	0.891	0.963	0.963

Note: Fitness indices: PNFI, PCFI, AGFI (>0.5), CFI, IFI (>0.9), RMSEA (>0.08), CMIN/df (>3 good, >5 acceptable).

Abbreviations: CFA, confirmatory factor analysis; CMIN/df, chi-squared/degrees of freedom ratio; RMSEA, root mean square error of approximation; PCFI, parsimonious comparative fit index; PNFI, parsimony normed fit index; AGFI, adjusted goodness-of-fit index; IFI, incremental fit index; CFI, comparative fit index.

Figure 1

Structure of the Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder: the model of first-order confirmation factor analysis.

Abbreviations: e, error; Q, question.

Following the first-order CFA, a separate assessment of the factors of the Treatment Adherence Questionnaire for Patients with Combat Post-traumatic Stress Disorder and the correlation between the constructs occurred. The second-order factor analysis was done to examine whether or not all the factors fitted the general concept of Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder. Table 4 presents the indices’ goodness of fit for the second-order CFA compared to the first-order model. Figure 2 shows the second-order structural model and the CFA.

Figure 2

Structure of Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder: modified model of the second-order confirmation factor analysis.

Abbreviations: e, error; TAQ, Treatment Adherence Questionnaire; Q, question.

Convergent and divergent validity

Assessing the AVE, MSV, and ASV in the first-order CFA, however, showed that the Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder did not have a good convergent and divergent validity, but AVE was .0.5 in the second-order CFA, thus convergent validity was confirmed.

Reliability

As shown in Table 5 all three factors extracted had acceptable internal consistency and CR (>0.7) (Table 5). Stability (test-retest) of the questionnaire was investigated using ICC. The mean scores of the questionnaire in the test and retest were 51.46±13.17 and 49.36±12.55, respectively. ICC was 0.923 (P<0.0001) 95% CI: 0.910–0.937 (Table 6).

Table 5

Convergent and divergent validity, internal consistency, and construct reliability of the Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder

Factor	α	Θ	Ω	CR	First-order			Second-order
					AVE	MSV	ASV	AVE
Maintenance of treatment (six item)	0.852	0.827	0.825	0.856	0.502	0.549	0.536	0.804
Follow-up and treatment contribution (five item)	0.843	0.811	0.823	0.846	0.526	0.523	0.511
Purposefulness and responsibility (six item)	0.838	0.820	0.840	0.845	0.480	0.549	0.524

Abbreviations: α, Cronbach’s alpha coefficient; θ, Theta coefficient; Ω, McDonald Omega coefficient; CR, construct reliability; ASV, average shared squared variance; MSV, maximum shared squared variance; AVE, average variance extracted.

Table 6

ICC, standard error of measurement, and smallest detectable change of the Treatment Adherence Questionnaire for Patients with Combat Post-Traumatic Stress Disorder

Factor	ICC (95% CI)	SEM	SDC
Maintenance of treatment	0.852 (0.823–0.878)	1.992	5.521
Follow-up and treatment contribution	0.842 (0.810–0.870)	1.788	4.956
Purposefulness and responsibility	0.871 (0.846–0.894)	1.961	5.435
Total questionnaire	0.923 (0.910–0.937)	3.630	9.753

Abbreviations: ICC, intra-class correlation; SEM, standard error of measurement; SDC, smallest detectable change.

As shown in Table 6, the rate of SEM and the SDC for the combat-related PTSD Treatment Adherence Questionnaire was 3.630 and 9.753, respectively.

Discussion

This study was aimed at the development and psychometric study of the Treatment Adherence Questionnaire in Iraqi war imposed PTSD veterans in Iran during 2016–2017. The Treatment Adherence Questionnaire for veterans with combat-related PTSD was developed using a concept analysis with the aid of a hybrid model qualitative study. It was designed on the basis of attributes and consequences identified in the concept of treatment adherence in this group of patients. First, 56 items were developed to measure this concept on the basis of the identified codes. Of these, 26 items were selected by a panel of experts and this was finally reduced to 17 items to be psychometrically investigated. The qualitative face validity of this questionnaire was first evaluated by experts whose opinions were applied to the questionnaire and finally, the questionnaire was completed for the target group. The items requiring modification were altered and reworded for clarification and betterment of the questionnaire content. Seif (2012) stated that “confirmation of the face validity refers to the acceptability and logicality of the test for the testees, so that lack of approval of this kind of validity makes the questionnaire incomprehensible to the testees”. This may negatively affect the method of responding to the questionnaire items, consequently reducing the motivation for answering the items.31 Polit (2012) also considers face validity of the questionnaire as one of the most important indices used to convince the target group to participate in the test which affects the results of the test.32 Considering the quantitative face validity in the present study, all the items had impact score greater than 3.5. When the impact score points of an item are estimated to be greater than 1.5, that item is rendered as appropriate. Hence, all the items were maintained in this phase. Regarding the qualitative measurement of content validity, some changes were applied to many items of the inventory to make it more objective and understandable.33 Polit (2012) considers content validity as measurement of a questionnaire necessary. She states that “although there is no totally uniform method of content validity investigation available yet, the use of experts’ opinion is necessary as a rule of thumb”.32 In this study, the experts’ opinions on the item characteristics, content, and questionnaire guide were exerted. The ratio of content validity of the items in this questionnaire was measured at two stages. In the first phase, using Lawshe table (ten experts), the items with CVR less than 62% were omitted from the questionnaire. The CVR of all items was reported as one in the second phase. This was the case while the acceptable value of CVR on the basis of Lawshe table was 0.99–5 experts.34 Therefore, it can be said that the presence of all the items is necessary in this questionnaire. In the present study, the CVI of each item ranged from 0.8–1 on the basis of the three criteria of relevance, clarity, and simplicity, while the total CVI for the three mentioned criteria was greater than 0.95. A CVI greater than 0.79 indicates the suitability of the item, while a CVI between 0.70 and 0.79 makes the item problematic and suspicious, thus requiring correction and revision.35 A content validity index less than 0.70 is unacceptable and any item with such an index value ought to be removed. The CVI of the whole questionnaire is rendered as appropriate if the value is greater than 0.9.32,33 Hence, the final version of the Treatment Adherence Questionnaire of war veterans with PTSD in Iran possesses the required content validity index.

The EFA of the three factors maintenance of treatment (six item), follow-up and treatment contribution (five item), and purposefulness and responsibility (six item) of the final version of Treatment Adherence Questionnaire for Iranian war veterans with combat-related PTSD revealed that these factors can collectively load almost 87.57% of the variance of the total score of the cognitive status test. Among these, the greatest values of the determined variance belonged, respectively to the first factor (30.00%) and second factors (29.57%). Maintenance of treatment and follow-up and treatment contribution are considered as two important factors in treatment adherence; so, considering the concepts of the items loaded in the first and second factors, it is justified that the greatest part of the variance is loaded by the first and second factors. Of course, the variance share of the other factors was greater than 5%. In EFA, one of the factors related to the judgment of construct validity of a questionnaire is the amount of variance determined by the identified factors.36 Polit (2012) asserts that the identified factors in an EFA must at least account for 60% of the total variance of the scores, while each factor should load at least 5% of the variance.32 Some scholars have mentioned the variance percentage of the whole factors to be more than 50%.37–39 Hence, based on EFA, the final version of Treatment Adherence Questionnaire for Iranian war veterans with combat-related PTSD is considered as a three-factor questionnaire with construct validity. In this study, to discover a more detailed model of structural equations, the second-order CFA was also performed. This method aimed to obtain a more significant method of data collection while assuming that the latent variables in the common variance are due to one or more higher-order factors and that the proposed scale has two orders.40 A high correlation between the first-order constructs displays that the latent variables do not completely act as an independent variable and the correlation between them reflects the presence of a more general construct in a secondary conceptual level, where the finest approach to the assessment of the structure is structural equation modeling, as it can identify the first-order constructs that were proposed as the latent variables.41 Some research suggested that the intended construct must first be created through first-order factor analysis, and the good fit of the conceptual construct then be determined for the assessment of the structural equation model using second-order factor analysis.42

The Cronbach’s alpha correlation coefficient was estimated for the whole questionnaire as 0.924. Houser (2013) sets the minimally significant reliability coefficient as 0.7 and believes a reliability coefficient between 0.7 and 0.9 to be moderate while a coefficient of 0.9 is strong.38 Nonetheless, Tappen (2010) believes that for newly developed questionnaires, a Cronbach’s alpha greater than 0.7 is acceptable.43 The questionnaire consistency coefficient in the test-retest was 0.867. Houser (2013) considers reliability coefficient of greater than 0.7 as satisfactory.38 Test-retest is one of the common methods of reliability measurement that explores the rate of consistency and repeatability of a test. The use of this method is common in studies on psychometric screening of questionnaires.44 Polit (2012) postulates that a reliability coefficient greater than 0.70 is satisfactory, while coefficients greater than 0.8 and 0.9 are very good and ideal, respectively.32,45 Given the correlation coefficients estimated in this study, it can be asserted that this questionnaire enjoys proper consistency and repeatability. The ICC rate of the questionnaire under study for the whole questionnaire and its factors was more than 0.8. To interpret this finding, it can be said that an ICC value less than 0.5 is weak, between 0.5 and 0.75 is moderate, between 0.75 and 0.90 is good, and a value greater than that is excellent.46 Hence, the ICC of the questionnaire possesses good consistency. The findings in this study suggest that the Treatment Adherence Questionnaire for Iranian war veterans with combat-related PTSD suffered from an SEM of 3.63 for the whole questionnaire. The SEM of the factors identified in EFA was between 1.7 and 1.59. Given that the total score of the questionnaire was 85, the identified SEM is small and acceptable. The minimal SEM of the questionnaire is very important because only changes that are greater than the estimated SEM of the questionnaire are clinically significant. Indeed, SEM quantifies the accuracy of the score of any individual.47 Agreement of the questionnaire must also be investigated along with SEM of the questionnaire. In the present study, the SDC for the whole questionnaire was 9.75, while this value was between 4.9 and 9.7 of the four factors identified in this questionnaire. The questionnaire agreement parameter is positive if the SDC is greater than the minimal important change. In fact, the SDC is the change that is real and is not due to questionnaire error.47 The SDC value can be estimated from various levels of confidence. In the present study, a 95% CI was used to calculate this index where the SDC was obtained as 9.75. This value implies that a change of more than eight points in the present questionnaire in the two phases resulted from the effects of intervention and not due to SEM of the questionnaire.

Limitation

This study had one limitation: all samples studied were male, because most of the veterans were male in the Iran-Iraq war. Therefore, it is suggested that in future studies, gender effects should also be considered.

Clinical implication

This questionnaire is applicable to programs such as health monitoring program of veterans with combat PTSD. The questionnaire designed in this study can be applied to determine the status of adherence to treatment for combat PTSD before or during the course of treatment in inpatients and outpatients. Also, by using this questionnaire, the impact of different interventions on adherence can be examined for treatment of these patients.

Future research

Identifying and measuring effective factors for adherence to treatment for combat PTSD can help in the process of educating the patient and the family, reforming the social support system, and modifying the behavior of therapists. It is also recommended that future studies focus on evaluating the diagnostic value of this questionnaire.

Generalizability

Although using a hybrid approach (eleven documents from other countries except Iran were analyzed) has increased the generalizability of this study, it seems necessary to evaluate its validity and reliability in other countries and cultures.

Conclusion

The final version of the Treatment Adherence Questionnaire for Iranian war veterans with combat-related PTSD included 17 items with approved face and content validity. On the basis of EFA, this questionnaire entailed three factors which were collectively able to load 87.57% of the variance in treatment adherence. The questionnaire consisted of 17 items in three components: maintenance of treatment (six item), follow-up and treatment contribution (five item), and purposefulness and responsibility (six item). Not only was the convergent and divergent of the first-order CFA not confirmed, but the convergent validity in the second-order CFA was also fulfilled. All items were rated on a Likert scale from 1 (not at all) to 5 (always), with minimum and maximum scores of 17 and 85, respectively. The higher the score, the greater was the degree of treatment adherence. Internal consistency correlation coefficient of the questionnaire was excellent. The minimal detectable change of the questionnaire was 9.753 and its SEM was 3.630. Consequently, the final version of Treatment Adherence Questionnaire for Iranian war veterans with combat-related PTSD may be applied as a valid and reliable questionnaire for measuring treatment adherence in these patients.