Development and Psychometric Evaluation of Coronary Artery Disease Treatment Adherence Scale.

INTRODUCTION: Poor treatment adherence reduces treatment effectiveness and increases healthcare costs. The promotion of treatment adherence necessitates valid and reliable tools to assess the effectiveness of adherence promotion strategies. This study was undertaken to develop and evaluate the psychometric properties of the Coronary Artery Disease Treatment Adherence Scale.
METHODS: This methodological study was conducted in two phases. In the first phase, a literature review was done and a 62-item scale was developed. In the second phase, different methods were used to assess the psychometric properties of the scale, namely its face validity, content validity, construct validity, internal consistency, and test-retest stability.
RESULTS: During face and content validity assessments, the number of items was reduced from 62 to 53. Moreover, in construct validity assessment via exploratory factor analysis, 18 items were deleted because their factor loading values were less than 0.3. The remaining 35 items were loaded on four factors, namely medication adherence, dietary adherence, exercise adherence, and healthy lifestyle adherence. The Cronbach's alpha values of the scale before and after factor analysis were 0.850 and 0.862, respectively. Its test-retest intraclass correlation coefficient was also 0.85.
CONCLUSIONS: The Coronary Artery Disease Treatment Adherence Scale is a valid and reliable tool and can be used in different healthcare settings for the assessment of treatment adherence among patients' with coronary artery disease. TRIAL REGISTRATION: Tehran University of Medical Sciences, identifier, 35456.

Introduction

Coronary artery disease (CAD) is the first leading cause of death [1] with 17.5 million cases in the world [2, 3] CAD mortality rate in the world and in developing countries is 25% and 32%, respectively [4]. Around 80% of CAD-induced deaths happen in low- and moderate-income countries [1]. CAD is also the first leading cause of death in Iran with a mortality rate of 39.3% [2].

There are different treatments for CAD, including pharmacological treatments, percutaneous coronary intervention, and coronary artery bypass surgery. However, these treatment modalities do not stop the processes of atherosclerosis and coronary artery stenosis [5, 6]. A study showed that in a 2-year follow-up period after coronary angioplasty, 30 to 40% of patients experienced recurrent angina, myocardial infarction, another angioplasty, or death [7].

The most significant factor behind treatment success is the patients’ adherence to treatments [8]. The World Health Organization Adherence Meeting in June 2001 defined adherence as “the extent to which the patient follows medical instructions” [9]. Treatment adherence can reduce CAD mortality rate by 11%. Contrarily, poor treatment adherence among CAD patients significantly increases mortality rate, rehospitalization rate, and healthcare costs [8]. Estimates show that in the United States of America, treatment non-adherence causes 125,000 cases of death annually and accounts for 33–69% of re-hospitalizations [10]. The World Health Organization reported that long-term treatment adherence among patients with chronic diseases is 50%. This rate is much lower in developing countries [9]. Another study showed that 13–60% of patients with myocardial infarction do not adhere to their treatments [11].

Assessment of treatment adherence is essential for planning treatments, ensuring their effectiveness, and deciding on future treatments. Such an assessment requires valid and reliable tools. However, there is no gold standard tool for adherence assessment and thus previous studies used different methods for the assessment [9]. The most commonly used adherence assessment tool is the Morisky Medication Adherence Scale [12]. This scale was developed by Morisky et al. in 1986 and translated into Persian by Negarandeh et al. [13]. Despite its popularity, this scale solely assesses adherence to medications and does not address other components of treatment adherence. Some studies on CAD patients also used researcher-made adherence assessment tools, which had been used without providing any reliable data on their development and psychometric evaluation. Other adherence assessment tools also have obvious shortcomings, such as variant scales for subscale scoring, which cause difficulties in answering items, analyzing and interpreting the collected data, and comparing subscale scores [14, 15]. Therefore, there is a great need for valid and reliable comprehensive tools for adherence assessment [16-18]. This study was undertaken to develop and evaluate the psychometric properties of the Coronary Artery Disease Treatment Adherence Scale (CADTAS).

Methods

This article resulted from a PhD dissertation in Nursing approved by Tehran University of Medical Sciences, identifier, 35456.

The study was conducted in two phases. In the first phase, scale development is carried out and in the second stage; judgment/quantification on scale items is performed (psychometric evaluation) (Fig. 1).

Fig. 1

Stage of development and psychometric evaluation of CADTAS

Phase 1. CADTAS Development

The first phase in the scale development is to generate and obtain items relevant to the question and the targeted population [19]. There are three methods for the initial item generation that included deductive, inductive, and a combination of the two. Deductive methods involve item generation based on an extensive literature review and pre-existing scales [20]. The most widely source of qualitative data is the literature review that produces tests with effectively unlimited item banks [19, 21]. An additional function of the literature review is to gain a more concrete understanding of the construct by identifying conceptual definitions of each dimension and relevant empirical items that describe each dimension [22].

CADTAS was developed deductively based on the existing literature. Accordingly, in order to determine the dimensions of treatment adherence and generate CADTAS items, we searched online databases such as ScienceDirect, Embase, MEDLINE, CINAHL, IranMedex, Magiran, and SID in order to retrieve adherence-related studies. Search terms were treatment adherence, CAD, ischemic heart disease, coronary angioplasty, and coronary artery bypass surgery. Studies were included in the analysis if they were directly related to treatment adherence, were literature review or original study, and had been published in Persian or English after 2000. This search protocol yielded 706 documents. Irrelevant studies were excluded and 352 studies and four questionnaires were kept. Then, the full texts of all retrieved documents were read and 75 documents directly related to CAD treatment adherence were selected. These 75 documents were reviewed and summarized to define CAD treatment adherence, determine its dimensions, and then, generate CADTAS items. We also considered societies’ recommendations for assessment of adherence.

After summarizing the documents, the concept of adherence to the treatment was explained and its dimensions were determined. We wrote the maximum possible items in each dimension.

Therefore, an item pool with 65 items was generated. The items were repeatedly reviewed, revised, and merged and finally, the primary CADTAS was generated with 62 items. Items were scored via a five-point Likert-type scale as follows: 1: “Never” (no treatment adherence); 2: “Rarely” (limited treatment adherence); 3: “Sometimes” (moderate treatment adherence); 4: “Often” (good treatment adherence); and 5: “Always” (strict treatment adherence). Higher scores represented closer treatment adherence. Some items were negatively worded in order to reduce the negative effects of haphazard responses to items.

Phase 2: CADTAS Psychometric Evaluation

In this phase, the validity and reliability of CADTAS were assessed. Validity assessment included the assessments of face, content, and constructs validity.

Assessment of Face Validity

Qualitative and quantitative methods were used for face validity assessment. In qualitative assessment of face validity, ten low-literacy CAD patients were randomly selected and interviewed face-to-face about the difficulty, appropriateness, and clarity of CADTAS items. Then, the items were revised to make them simpler and clearer.

Quantitative assessment of face validity was done via calculating impact score for each item. Ten randomly selected CAD patients rated the importance of each CADTAS item using a five-point rating scale as follows: 1: Not important; 2: Slightly important: 3: Somehow important; 4: Important: and 5: Very important. Impact score for each item was calculated using the following formula, “Impact score = Frequency (%) × Importance”. “Frequency” in the formula was the number of patients rated the item 4 or 5, while “Importance” was the mean score of the item on the 1–5 rating scale. Items were kept in CADTAS if they obtained an impact score of 1.5 or more [23, 24].

Assessment of Content Validity

Assessment of content validity was also performed via both qualitative and quantitative methods. In the qualitative method, ten experts in instrument development, cardiology, cardiac surgery, cardiac nursing, and treatment adherence evaluated and commented on the wording, essentiality, importance, appropriate placement, and scoring of the items. Then, items were revised using their comments. Quantitative assessment of content validity was performed by calculating content validity ratio (CVR) and content validity index (CVI). CVR was calculated to ensure that the most important and the most correct items had been selected for treatment adherence assessment. Accordingly, 14 experts in cardiology, cardiac surgery, and nursing evaluated the essentiality of the items via a three-point scale with the following three points: “Essential”, “Useful but not essential”, and Not essential”. Then, the CVR of each item was calculated via the CVR formula, i.e., CVR = (Ne – (N/2))/(N/2), where N stands for the total number of experts and Ne stands for the total number of experts rated the intended item as essential. Lawshe noted that for 14 experts, items with CVR values of 0.51 and more are appropriate [25].

CVI was calculated to determine whether the items appropriately measure treatment adherence. The same 14 experts rated the simplicity, relevancy, or specificity, and clarity of each item on a four-point rating scale. Then, the number of experts rated an item 3 or 4 was divided by the total number of experts in order to calculate the CVI of that item. Items with a CVI of more than 0.79 were kept, while items with a CVI of 0.70–0.79 were revised and items with a CVI of less than 0.70 were removed. Finally, the average CVR and CVI were calculated for the whole CADTAS.

Primary Assessment of Reliability

After the assessments of face and content validity and before the assessment of construct validity, the reliability of CADTAS was assessed through the internal consistency method. Accordingly, Cronbach’s alpha of the scale was calculated using the data collected from 30 CAD patients. The minimum acceptable Cronbach’s alpha was considered to be 0.70 [26].

Assessment of Construct Validity

Construct validity can be assessed with the use of exploratory factor analysis (EFA), confirmatory factor analysis (CFA), or with convergent, discriminant, predictive/ nomological, criterion, internal, and external validity. The group of extraction and rotation techniques to identify latent constructs is referred to as common factor analysis or exploratory factor analysis [20, 22].

As CADTAS was a new tool and we had no assumption about its dimensions, assessment of its construct validity was performed using exploratory factor analysis. The number of necessary individuals for factor analysis is 5–10 individuals per item [27]. Therefore, we recruited six CAD patients per CADTAS item. Inclusion criteria were a definite diagnosis of CAD via coronary angiography, an age of 18 or higher, and receiving CAD treatments for at least 2 months. In order to select a sample of maximum variation, patients were recruited from three teaching hospitals and four clinics in Zanjan, Iran. Sampling was done conveniently. Recruited patients individually completed CADTAS. The data were entered into the SPSS software (v. 24) for factor analysis. Initially, the Kaiser-Meyer-Olkin (KMO) test was run to ensure the adequacy of the selected sample. KMO test value can range from 0 to 1—the higher the test value, the more accurate the factor analysis. The minimum acceptable value is 0.70. Besides, Bartlett’s test of sphericity was run to determine factor analysis suitability by testing whether the correlations among CADTAS items were strong enough so that the items could be clustered and also by testing whether the correlation matrix was significantly different from zero value.

Factor extraction in factor analysis was done via the principal component analysis method. The results of this method are presented as a matrix of each factor loading on each item. In fact, factor loading represents the correlation of an item with a factor and is used to maintain an item in a factor or remove an item from a factor after factor rotation. There are different methods for determining the number of factors. In this study, the number of factors was determined based on the variance of each item, eigenvalue, and scree plot. Then, varimax rotation was used to simplify the factors and make them interpretable. Extracted factors were labeled based on the items loaded on them. The minimum factor loading for keeping an item in the allocated factor was 0.3.

Assessment of Reliability

The internal consistency and the test–retest stability methods were used for the assessment of reliability. Stability, also called test–retest reliability, is the degree to which scores on the same test are consistent over time. In other words, this type of reliability provides evidence that scores obtained on a test at one time (test) are the same or close to the same when the test is re-administered some other time (retest). The more similar the scores on the test over time, the more stable the test scores. A common, minimally accepted sample size for a correlational study is 30 participants. If the variables correlated have low reliabilities and validities, a bigger sample is necessary. The procedure for determining test–retest reliability is quite simple: after administrating the test to an appropriate group and passing some time- about 2 weeks or more- administer it again and calculate the correlation between two series of data [28, 29].

Accordingly, 30 randomly selected patients completed CADTAS. Collected data were used to calculate Cronbach’s alpha. Minimum acceptable alpha value was 0.7 [30]. Two weeks later, the same 30 patients recompleted CADTAS. Then, intraclass correlation coefficient was calculated to assess stability.

Compliance with Ethics Guidelines

All procedures performed in studies involving human participants were in accordance with the Ethics Committee of the Tehran Faculty of Nursing and Midwifery with No: IR.TUMS.FNM.REC.1396.2689 and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Results

Our literature review revealed that the meaning of the adherence concept has changed over time. Initially, it solely meant following a physician’s medical orders. Accordingly, patients were inactive and accountable for treatment failure. However, the concept of adherence is currently so comprehensive that it requires patients to get actively involved in treatments and self-care. In other words, treatment plans are not imposed on patients; rather, they are designed, implemented, and evaluated via patients’ active involvement and patient–professional interactions. In this viewpoint, adherence is determined by different factors and is considered to be the responsibility of patients (who act), healthcare professionals (who facilitate action), and social context (in which the action is performed) [8, 31, 32]. Currently, adherence is considered as the extent to which patients’ behaviors comply with healthcare professionals’ recommendations and instructions. Our literature review revealed that treatment adherence among patients with CAD consists of the four dimensions of adherence to dietary regimen, adherence to healthy lifestyle, adherence to medication therapy, and adherence to physical activity.

Item Generation

After identifying the four dimensions of treatment adherence based on the existing literature, the primary item pool was generated with 65 items. Items were repeatedly revised and reduced to 62.

Psychometric Evaluation

Qualitative assessment of face validity was done by four female and six male patients with primary and secondary educations. In quantitative assessment of face validity, four items were removed due to an item impact score of less than 1.5.

In qualitative assessment of content validity, changes were made to the items and three items were removed based on the comments of the ten experts. Moreover, in the quantitative assessment of content validity, two items with a CVR of less than 0.51 were deleted. All remaining 53 items had a CVI of more than 0.79. The average CVR and CVI of the whole scale were 0.886 and 0.977. Table 1 shows the number of items deleted during the different steps of psychometric evaluation.

Table 1

Items deleted in different steps of psychometric evaluation

Steps of psychometric evaluation	Number of items deleted	Number of remaining items
Primary assessment	3	62
Quantitative face validity assessment	4	58
Qualitative content validity assessment	3	55
Quantitative content validity assessment via CVR	2	53
Quantitative content validity assessment via CVI	0	53
Construct validity assessment via factor analysis	18	35

After confirming face and content validity, primary reliability assessment was performed via the internal consistency method. Cronbach’s alpha of the 53-item CADTAS was equal to 0.85.

Construct validity of CADTAS was assessed via factor analysis. We recruited 318 CAD patients (six CAD patients per CADTAS item). Table 2 shows demographics characteristics of CAD patients recruited in evaluating construct validity and primary reliability assessment. KMO test value was 0.809, indicating that the recruited sample of patients was adequate enough for factor analysis. Moreover, the significant results of Bartlett’s test of sphericity showed strong correlations among items (χ2 = 4527.762; df = 528; P < 0.001), implying the suitability of factor analysis.

Table 2

Demographic and clinical characteristics of CAD patients recruited for evaluating construct validity, primary reliability of CADTAS

Variables		Frequency	%
Age	< 50 years old	51	16
	> 50 years old	267	84
Sex	Male	216	67.9
	Female	102	32.1
Level of education	Reading and writing	154	48.4
	Primary school	116	36.5
	Secondary school	28	8.8
	High school	20	6.3
Marital status	Single	5	1.6
	Married	313	98.4
Occupation	Housewife	101	31.8
	Laborer	40	12.6
	Clerk	33	10.4
	Self-employed	90	28.3
	Retired	54	17
Adequacy of income	Weak	40	12.5
	Average	127	39.9
	Good	151	47.5
Number of risk factors	One	170	53.5
	Two	119	37.4
	Three and more	29	9.3
Duration of the disease	Six months to 1 year	16	5
	1–2 years	263	83.7
	Older than 2 years old	39	12.3
Severity of CAD	SYNTAX Score ≤ 22	84	26.4
	SYNTAX Score = 23–32	104	32.7
	SYNTAX score ≥ 33	130	40.9
Percent of ejection fraction	< 35%	101	31.8
	35–49%	119	37.4
	> 50	98	30.8
Number of medications	< 4	98	30.8
	Apr	157	49.3
	> 7	65	19.9
Type of treatment	CABG	100	31.5
	PCI	154	48.4
	Medication follow-up	64	20.1

Factor number was determined via scree plot, eigenvalue, and factor rotation. Visual assessment of scree plot showed that 3–5 factors can be considered for CADTAS (Fig. 1). Based on eigenvalues, 17 factors with eigenvalues greater than 1 were identified, which collectively explained 63.5% of the total variance. The 17-factor structure was tested using different factor rotations. Finally, based on scree plot, the characteristics of the items, and factor rotation, we decided on a four-factor structure, which accounted for 45% of the total variance of treatment adherence. This four-factor structure had the highest level of intra-factor homogeneity and consistency. The minimum acceptable factor loading was 0.3. In the four-factor structure, 18 items had a factor loading of less than 0.3. These 18 items were deleted and therefore, the number of items reduced from 53 to 35. The four extracted factors were named based on their items and were compared with the four dimensions of treatment adherence which had been identified in the literature review. The four extracted factors included medication adherence (11 items), dietary adherence (13 items), exercise adherence (four items), and healthy lifestyle adherence (seven items). The five possible answers to CADTAS items were “Always” (scored 5), “Often” (scored 4), “Sometimes” (scored 3), “Rarely” (scored 2), and “Never” (scored 1).

The negative effects of haphazard responses are reduced through the negative wording of some items. The total CADTAS score is 35–175, with higher scores standing for closer treatment adherence among CAD patients.

Table 3 shows characteristics of CAD patients recruited for evaluating final reliability of CADTAS. Final reliability assessment indicated that the Cronbach’s alpha values and the intraclass correlation coefficients of CADTAS and all its four factors were greater than 0.70 and 0.80, respectively (Table 4).

Table 3

Demographic and clinical characteristics of CAD patients recruited for evaluating final reliability assessment of CADTAS

Variables		Frequency	%
Age	< 50 years old	5	16.7
	> 50 years old	25	83.3
Sex	Men	20	66.7
	Women	10	33.3
Level of education	Reading and writing	9	30
	Primary school	16	53.3
	Secondary school	3	10
	High school	2	6.7
Marital status	Single	0	0
	Married	30	100
Occupation	Housewife	9	30
	Laborer	4	13.3
	Clerk	5	16.7
	Self-employed	3	10
	Retired	9	30
Adequacy of income	Weak	4	13.3
	Average	7	23.3
	Good	19	63.3
Number of risk factors	One	9	30
	Two	10	33.3
	Three and more	11	36.7
Duration of the disease	Six months to 1 year	2	6.7
	1 to 2 years	21	70
	Older than 2 years old	7	23.3
Severity of CAD	SYNTAX score ≤ 22	9	30
	SYNTAX score = 23–32	14	46.7
	SYNTAX score ≥ 33	7	23.3
Percent of ejection fraction	< 35%	11	36.7
	35–49%	11	36.7
	> 50	8	26.6
Number of medications	< 4	9	30
	April 7	11	36.7
	> 7	19	33.3
Type of treatment	CABG	17	56.6
	PCI	11	36.7
	Medication follow-up	2	6.6

Table 4

CADTAS dimensions and their Cronbach’s alpha values and test–retest intraclass correlation coefficients

CADTAS dimensions	Items	Cronbach’s alpha	Intraclass correlation coefficient	P value
Dietary adherence	13 (1–13)	0.823	0.81	< 0.001
Healthy lifestyle adherence	7 (14–20)	0.746	0.83	> 0.001
Medication adherence	11 (21–31)	0.855	0.80	> 0.001
Exercise adherence	4 (32–3)	0.748	0.82	< 0.001
Total CADTAS	35 (1–35)	0.862	0.85	< 0.001

Discussion

This study aimed to develop and evaluate the psychometric properties of CADTAS. The final version of CADTAS has 35 items in four dimensions. CADTAS is completed in around 15 minutes via self-administration and hence, it is an almost simple tool for treatment adherence assessment in different settings, including hospitals, primary healthcare centers, clinics, rehabilitation centers, and home settings.

The findings of this study showed that CADTAS has acceptable face validity, content validity, construct validity, and reliability. It is a practical instrument for measuring patient adherence to treatment.

The first dimension, dietary adherence, is comprised 13 items. Diet has a fundamental role in development and prevention of cardiovascular disease (CVD). Although the traditional research focused on single nutrients or foods, recent studies of dietary patterns have allowed for reflection on both the complexity and the synergies of food and nutrient intake [33]. In an attempt to validate an instrument measuring dietary adherence, Huijbregts et al. [34], based on WHO international dietary guidelines, developed the Healthy Diet Indicator (HDI).

The second dimension was healthy lifestyle adherence, which comprises seven items. There is a positive correlation between modifiable risks factors such as smoking and the harmful use of alcohol with coronary heart disease. Existence of these items are very important because most patients with cardiovascular disease continue to smoke after acute myocardial infarction, exposing themselves to a 50.0% increased risk of recurrent coronary events among nonsmokers [35].

The third dimension, medication adherence, consists of six items. The most popular scale applied for the assessment of adherence is the Morisky Medication Adherence Scales (MMAS-4 and MMAS-8). This dimension of CADTS has a good Cronbach’s alpha value (0.855), the same as the Morisky Medication Adherence Scale (0.83). MMAS against the CADTAS not allow to comprehensive assessment of adherence behaviors. Consequently limit developing interventions to improving adherence on the results of this scale [36, 37].

The fourth dimension of the instrument entails exercise adherence, which comprises four items. Physical activity includes all forms of exercise such as walking, cycling, active play, work-related activity, active recreation and organized sport, dancing, gardening, or playing active games. Being physically active can reduce the risk of many chronic conditions such as coronary heart disease[38]. This dimension of CADTS has better intraclass correlation coefficient (ICC) reliability (0.82) than the Physical Activity Scale for the Elderly (PASE), which is widely used in epidemiological studies for assessing the physical activity level of the elderly (0.75) [39].

Adherence to treatment and active participation of the patient in the therapeutic process is beneficial for both individuals and communities [36]. Since studies have shown that half of the patients with chronic disease do not adhere to treatment, therefore, the assessment of patient adherence to treatment plays a vital role in the description and verification of the therapeutic process [40-42]. Due to the lack of valid and reliable tools, most previous studies assessed treatment adherence using laboratory techniques such the measurement of serum cholesterol, serum levels of mediations, and so on. However, these techniques are invasive and costly, do not provide comprehensive information about treatment adherence, and cannot be used in all healthcare settings [43]. The newly developed Medication Event Monitoring System is also very costly and just assesses medication adherence. Another pitfall of this system is its unsuitability in case of polypharmacy.

Previous studies have emphasized that there is an urgent need for studies aimed at a careful and standardized clinically applicable scale to ensure identification of patients’ adherence [36, 44]. For response to this need, we developed and have done psychometric evaluation of CADTAS. It can be completed personally by patients in around 15 min without imposing undue financial burden on them.

Due to the fact that CADTS has been developed and validated, clinicians such as physicians and nurses, can use this scale for assessment, identify related factors, and then design interventions on adherence to the treatment.

Limitations

The study was conducted in Iran. This could be a limitation to generalize the results to other regions where disease population/education and culture are different.

The test re-test reliability was done on 30 patients with 2-week interval. Short period between tests can have a memory effect. Also, reliability could have been done on a large sample.

Conclusions

CADTAS is a valid and reliable tool for the assessment of CAD patients’ treatment adherence. Due to its simple scoring system, comprehensiveness, and easy applicability in different healthcare settings, CADTAS can be used by different healthcare providers, including nurses, for treatment adherence assessment among patients with CAD.

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 158 kb)