The validity of the MacNew Quality of Life in heart disease questionnaire.

BACKGROUND: A previous review suggested that the MacNew Quality of Life Questionnaire was the most appropriate disease-specific measure of health-related quality of life among people with ischaemic heart disease. However, there is ambiguity about the allocation of items to the three factors underlying the MacNew and the factor structure has not been confirmed previously among the people in the UK.
METHODS: The MacNew Questionnaire and the SF-36 were administered to 117 newly admitted patients to a tertiary referral centre in Northern Ireland. All patients had been diagnosed with ischaemic heart disease.
RESULTS: A confirmatory factor analysis was conducted on the factor structure of the MacNew and the model was found to be an inadequate fit of the data. A quantitative and qualitative analysis of the items suggested that a five factor solution was more appropriate and this was validated by confirmatory factor analysis. This new structure also displayed strong evidence of concurrent validity when compared to the SF-36.
CONCLUSION: We recommend that researchers should submit scores obtained from items on the MacNew to secondary analyses after being grouped according to the factor structure proposed in this paper, in order to explore further the most appropriate grouping of items.

Background

The goals of health and social care interventions in the field of ischaemic heart disease are placing increasing emphasis on patients' quality of life as well as quantity of life and an improvement in health-related quality of life (HRQoL) is often a stated goal of many cardiac rehabilitation programmes and secondary prevention strategies. The identification or measurement of change in HRQoL is, therefore, an important aspect of the evaluation of secondary prevention programmes in heart disease. Researchers have used several generic and disease-specific instruments in their attempts to index HRQoL – a review of the instruments used to measure HRQoL among people with ischaemic heart disease identified four generic instruments and nine disease-specific instruments [1]. This review suggested, on the basis of the best available psychometric evidence, that the most appropriate generic and disease-specific measures of HRQoL for use among people with ischaemic heart disease were the Short Form 36 (SF-36) [2] and version two of the Quality of Life after Myocardial Infarction questionnaire (now referred to as the MacNew Quality of Life Questionnaire) [3], respectively.

Much has been written about the psychometric properties of the SF-36 and the results concerning the factor structure of this instrument appear to be fairly consistent (see Ware et al. [4]). Less psychometric information exists on the MacNew.

The MacNew is a revised version of the Quality of Life after Myocardial Infarction Questionnaire (QLMI) [5] and there is much difference between the two versions in terms of their suggested factor structure, even though the items on the two versions of this questionnaire are similar. The original QLMI questionnaire contained 26 items which were divided into five underlying scales or factors: self-esteem, restriction, fatigue, emotional function and confidence. The MacNew has 27 items reduced to three factors: social functioning, physical functioning and emotional functioning. The assignment of items to the factors underlying the MacNew was done on the basis of a factor analysis conducted by Valenti et al. [3]. They suggested that two items in the original QLMI should be excluded and another item modified and they included three new items in the MacNew. However, the results of the factor analysis presented by Valenti et al. raise a couple of issues. Firstly, the three new items included in the MacNew increased the percentage variance explained by all the items from 65.8% to 66.5%. The benefit of the small increase in explanatory power appears to be outweighed by the lengthening of the questionnaire. Secondly, the attribution of items to factors is ambiguous. Certain items appear to be allocated to factors that do not have the highest factor loading for those items and with which there is a poor conceptual relationship. Furthermore, the factor structure of the MacNew has not been examined within the UK population.

It is essential that HRQoL instruments are valid and so there is a need to examine the content of the MacNew scales, in order to confirm (or not) the construct validity of this questionnaire. Therefore, the aim of the present paper was to test the factor structure of the MacNew among a group of patients from the UK with ischaemic heart disease.

Methods

Newly admitted patients to a tertiary referral centre in Northern Ireland who were diagnosed as having ischaemic heart disease were identified by the first author in consultation with the wards' nursing staff. A total of 119 patients, all of whom were assessed by a consultant cardiologist, were asked, at hospital, for their consent to participate in the study. Only two patients (1 male, 1 female) refused to participate. Therefore, the MacNew and the SF-36 were administered in interview format to 117 patients (see Table 1) either in hospital (101/117) or at a hospital-based cardiac rehabilitation class (16/117).

Table 1

Characteristics of Sample

Age (mean(sd))	60.61(10.16)
BMI (mean(sd))	27.2(4.4)
Length of diagnosis of IHD (median(range))	6 months(2 days to 42 years)
Sex	84 males; 33 females
Reason for admission to hospital	37 myocardial infarction; 63 angina; 17 other
Canadian Cardiovascular Society Classification of Angina	43 class 0; 12 class I; 21 class II; 16 class III; 25 class IV

The MacNew questionnaire consists of 27 items, each with a 7 point response scale ranging from "all of the time" to "some of the time". Evidence exists to support the reliability, validity and sensitivity of this instrument and its original version (the QLMI) [6]. The SF-36 consists of 36 items which are divided into eight dimensions: physical functioning, social functioning, general health, role limitations due to physical problems, role limitations due to emotional problems, bodily pain, vitality, and mental health. The psychometric properties of the SF-36 have been investigated in a variety of patient groups [2] and there is strong evidence for the reliability and validity of this instrument when used among people with ischaemic heart disease [1].

The patients who participated in the study were similar to patients from the UK with ischaemic heart disease in previous studies in terms of their age [7], sex [8] and BMI [9]. The distribution of patients between the different classes of the CCSCA suggested that our group of patients had less severe angina than a group of patients undergoing angioplasty [8]. This conforms to expectations.

Results

The summary statistics for each scale are given in Table 2. All scales were transformed so that they had a possible range of 0 to 100, with a higher score indicating a better level of functioning or HRQoL. The transformation of scores was completed using the formula suggested for transforming the SF-36 scores:

Table 2

Summary statistics for all scales.

Scale	Mean	Standard deviation	Median
MacNew Social	70.00	22.01	73.33
MacNew Physical	70.49	18.22	73.81
MacNew Emotional	66.89	19.40	69.70
SF-36 General health	66.86	19.70	72.00
SF-36 Physical functioning	66.41	27.00	75.00
SF-36 Role physical	37.61	46.50	0.00
SF-36 Role emotional	84.33	35.44	100.00
SF-36 Social functioning	63.78	32.22	75.00
SF-36 Bodily pain	70.23	31.94	82.00
SF-36 Vitality	42.65	24.04	40.00
SF-36 Mental health	70.56	22.51	76.00

(((actual raw score-lowest possible raw score)/possible raw score range) * 100).

The MacNew scores were transformed using this formula to facilitate comparisons between questionnaires. Patients had moderate to high scores on all scales except the SF-36 scales of vitality and role physical.

An initial confirmatory factor analysis on the MacNew was conducted, using LISREL 8.3 [10], in order to test the factor structure proposed by Valenti et al. Confirmatory factor analysis assesses the validity of a pre-specified factor model. In other words, this statistical procedure assesses whether or not (on the basis of the relationships between the items on a questionnaire) the suggested method of combining items into groups (which are supposed to represent underlying factors) is reasonable. The LISREL programme produces several goodness of fit indices, which are designed to provide a summary answer to the assessment just described. In essence these indices inform the researcher whether or not the suggested factor model is a good fit of the existing data. The goodness of fit of this model was assessed using the Root Mean Square Error of Approximation (RMSEA), as an alternative to chi-square. The chi-square statistic has been associated with sample size problems and distributional misspecifications [11,12], hence the increasing reliance on alternative fit indices. One of the most popular and robust fit statistics used in conjunction with the chi-square is the RMSEA. Unlike the chi-square it is not affected by sample size. An adequately fitting model will have an RMSEA index between 0.00 and 0.06 with confidence intervals between 0.00 and 0.08 [11].

The factor structure proposed by Valenti et al. had a RMSEA = 0.092 (CI: 0.081 to 0.100). Therefore, this proposed structure did not provide a good fit of the data. Consequently, an exploratory factor analysis was conducted, in order to provide a suggestion about an adequate factor model. The exploratory factor analysis suggested six factors.

A content analysis of these factors was undertaken and the following points were noted. Firstly, factor 6 contained only one item – item 21 ("how often have you felt unsure as to how much exercise or physical activity you should be doing?"). During administration of the questionnaire, it had been recorded by the interviewer that this item did not appear to tap into the way heart disease had impacted on the person's HRQoL. Instead, it seemed to assess knowledge about the restrictions imposed by ischaemic heart disease. The factor analysis confirmed that this question did not relate well to any of the other items. Therefore, it was felt that this item should be excluded.

Secondly, there were several items that loaded heavily on more than one factor and a decision about how to deal with these items was based on the content analysis. It made sense, in terms of content validity, for these items to be grouped under the factor onto which they loaded heaviest. Therefore, to improve the content validity of the factors, it was decided that: item 12 should be grouped under factor 5; item 8 should be grouped under factor 3; item 13 should be grouped under factor 4. Item 23 loaded heavily on two factors and an analysis of its content suggested that it could sensibly be grouped with both of these factors. However, there was a problem with items 6 and 14. These items loaded onto factors that contained items which appeared to be dissimilar in content. In fact, both of these items appeared to be closer to all of the items that loaded onto factor 3.

The result of this qualitative exploration is a proposed five factor structure, as shown in bold in Table 3. The factors have been labelled emotion, restrictions, physical symptoms, perception of others, and social functioning. This proposed structure was submitted to a confirmatory factor analysis and the resulting model is displayed in Figure 1. The RMSEA = 0.049 (CI: 0.032 to 0.064) of this proposed model, which suggested that the fit was adequate.

Table 3

Exploratory Factor analysis of the MacNew items.

Item No.	Emotion	Restriction	Symptoms	Perception of others	Social
3. Confident	0.76
4. Down in the dumps	0.73
2. Worthless	0.73
15. Lack self-confidence	0.73
18. Frightened	0.71
10. Tearful	0.68
5. Relaxed	0.68
1. Frustrated	0.64
7. Happy	0.60
23. Burden on others	0.56			0.51
6. Worn out	0.44				0.36
17. Limited in exercise		0.89
26. Physically restricted		0.88
20. Restricted		0.88
12. Social activities		0.47		0.39	0.46
19. Dizzy			0.74
16. Aching legs			0.74
9. Short of breath			0.72
8. Restless	0.44		0.48
22. Family overprotective				0.79
11. More dependent				0.66
13. Others' confidence			0.42	0.48
14. Chest pain					0.74
25. Unable to socialise		0.42		0.41	0.55
24. Felt excluded	0.35	0.37			0.42
Total variance explained	21.62%	13.70%	9.90%	9.29%	7.33%	Total = 61.84%

N.B. All factor loadings < 0.35 have been omitted to aid clarity

Figure 1

Confirmatory factor analysis of suggested five factor structure underlying the QLMI-2. The circles represent the unobserved latent factors, the boxes represent the observed measures (questionnaire items), the lines pointing from the factors to the observed measures represent the factor loadings (when squared this value is the percentage of variation in the observed measure which can be explained by the factor), the shorter lines pointing to the observed measures represent the measurement error which is presumed to be random in nature, and the curved arrows joining each factor represent the correlation between factors.

The concurrent validity of this proposed factor structure of the MacNew was compared to the structure proposed by Valenti et al. by calculating the extent to which the different factors correlated with the SF-36 scales (see Table 4). The points where we expected the strongest relationships are in bold. In most cases, this expectation was confirmed, but there appears to be greater discrimination between the factors of our proposed structure than between the three factors suggested by Valenti et al. For example, Valenti's social and physical functioning scales both correlate strongly with the SF-36 social and physical functioning scales. In contrast, only the proposed physical symptoms scale correlates highly with the SF-36 physical functioning scale and only the proposed restrictions scale correlates highly with the SF-36 social functioning scale.

Table 4

Correlations between MacNew factors and SF-36 scales.

	Scales	GH	PF	RP	RE	SF	BP	VT	MH
Proposed factors of MacNew	Emotion	0.55	0.54	0.42	0.57	0.51	0.27	0.57	0.85
	Restrictions	0.18	0.39	0.56	0.21	0.55	0.33	0.36	0.30
	Physical symptoms	0.54	0.68	0.51	0.32	0.53	0.60	0.70	0.50
	Perception of others	0.30	0.33	0.37	0.41	0.46	0.32	0.28	0.52
	Social functioning	0.40	0.50	0.59	0.28	0.83	0.37	0.47	0.43
Existing factors of MacNew	Social functioning	0.37	0.42	0.48	0.39	0.64	0.36	0.34	0.51
	Physical functioning	0.36	0.56	0.56	0.25	0.62	0.48	0.50	0.39
	Emotion	0.48	0.47	0.46	0.44	0.46	0.34	0.63	0.74

All correlations above an absolute value of 0.16 were statistically significant at the 5% level. GH = General health; PF = Physical functioning; RP = Role physical; RE = Role emotional; SF = Social functioning; BP = Bodily pain; VT = Vitality; MH = Mental health

Discussion

The structure of the original QLMI questionnaire was altered by Valenti et al. [3], but this new structure has not been tested or confirmed within the UK population. A confirmatory factor analysis demonstrated that the factor structure proposed by Valenti et al. did not adequately fit our data and so we explored the data in order to determine the "best" factor model for the items on the questionnaire. Through a combination of quantitative and qualitative approaches one item was deleted and the remaining items were grouped into five factors – emotion, restrictions, physical symptoms, perception of others and social functioning. This proposed factor structure was confirmed as an adequate model by means of confirmatory factor analysis. This factor structure bore some similarity to the grouping of items on the original QLMI. For example, our restrictions and symptoms factors are similar to the restrictions and symptoms factors respectively, suggested by Hillers et al. [5], who developed the original QLMI and our emotion factor contains all the items on their original emotion factor. Hillers et al. did not conduct a factor analysis on their suggested factor structure, but obtained their structure by an extensive programme of item analysis and reduction. The analysis reported in this paper goes some way towards confirming the benefit and usefulness of their approach.

Furthermore, grouping the items into the structure suggested by the analysis reported here increased the correlations between the MacNew scales and relevant scales on the SF-36, when compared to the grouping of items suggested by Valenti et al. Thus, the proposed grouping of items suggested presently has stronger evidence of concurrent validity than that suggested by Valenti et al.

However, these findings need to be confirmed by new data and we would encourage others to re-analyse data obtained from the MacNew, using our proposed factor structure in order to test the model suggested. This is a worthwhile exercise to clarify the validity of this instrument, so that clinicians will be confident that they are accessing relevant and accurate information about how a patient's life is affected by ischaemic heart disease. Patients are likely to benefit if this information is accessible to clinicians as this will ensure that the specific factors which contribute to a person's HRQoL can be monitored and considered when treatment protocols are drafted, thereby enabling comprehensive and holistic treatments.

Conclusion

Improvements in the health-related quality of life of people with heart disease is an important consideration in the assessment of treatment effectiveness [13]. The MacNew questionnaire is a reliable and valid method of assessing changes in health-related quality of life among this population. This exploratory study indicates that the grouping of items on this questionnaire may be modified and improved to provide more useful indicators of health-related quality of life.

Authors' contributions

MDempster assisted in the design of the study, conducted the data collection, assisted with the data analysis and drafted the manuscript. MDonnelly assisted in the design of the study and the drafting of the manuscript. COL assisted with the data analysis and the drafting of the manuscript. All authors read and approved the final manuscript.