The MacNew Heart Disease health-related quality of life instrument: a summary.

BACKGROUND: The measurement of health, the effects of disease, and the impact of health care include not only an indication of changes in disease frequency and severity but also an estimate of patients' perception of health status before and after treatment. One of the more important developments in health care in the past decade may be the recognition that the patient's perspective is as legitimate and valid as the clinician's in monitoring health care outcomes. This has lead to the development of instruments to quantify the patients' perception of their health status before and after treatment.
METHODS: We review evidence supporting the measurement properties of the MacNew Heart Disease Health-related Quality of Life [MacNew] Questionnaire which was designed to evaluate how daily activities and physical, emotional, and social functioning are affected by coronary heart disease and its treatment.
RESULTS: Reliability was demonstrated by using internal consistency and the intraclass correlation coefficients for the three domains in the Dutch, English, Farsi, German, and Spanish versions of the MacNew. With internal consistency and intraclass correlation coefficients =>0.73, reliability is high. Validity of the MacNew was examined with factor analysis and three core underlying factors, physical, emotional, and social, were identified, explaining 63.0 - 66.5% of the observed variance and replicated in the translations with psychometric data. Construct validity of the MacNew was further demonstrated by extensive substantiation of the logical relationships, defined a priori, between items and other comparison tools. The MacNew is responsive and sensitive to changes in HRQL following various interventions for patients with heart disease with 11 of 13 effect size statistics >0.80. Taking an average of 10 minutes or less to complete, the respondent-burden for the MacNew is low and its acceptability is demonstrated by response rates of over 90%. Normative data are available for patients with myocardial infarction, angina, and heart failure in the English version.
CONCLUSION: The MacNew may be a valuable tool for assessing and evaluating health related quality of life in patients with heart disease.

Introduction

One of the more important developments in health care in the past decade may be the recognition that the patient's perspective is as legitimate and valid as the clinician's in monitoring health care outcomes [1,2]. The added value in better understanding the impact of disease from the patient's perspective has led to the development of instruments to quantify the patients' perception of their health status before and after treatment. Traditional outcomes of treatment for coronary heart disease such as mortality and objective physiological or exercise tests give little information about the impact of either the condition or treatment from the patient's perspective.

Health-related quality of life [HRQL] has been defined by various authors; examples include HRQL as a measure of the patient's perspective representing the "functional effect of an illness and its consequent therapy upon a patient, as perceived by the patient" [3] and that proposed by Patrick and colleagues [4] as "the value assigned to duration of life as modified by the impairments, functional states, perceptions, and social opportunities that are influenced by disease, injury, treatment, or policy". HRQL is now considered an important outcome measure in investigations of therapeutic interventions for patients with chronic conditions such as cancer and heart disease, in epidemiological studies and in patient care, representing a paradigm shift in the assessment of efficacy and effectiveness [5]. This paradigm shift is partly a result of the introduction of a "biopsychosocial" perspective in medicine [6]. Coronary heart disease is the major cause of disability in many developed regions and by 2020 is forecast to be the major cause of disease burden world-wide [7]. Patients who survive an acute coronary event may live for an extended period of time but often may be disabled, helping to drive the increased attention in HRQL outcomes in studies of coronary heart disease [5]. As an outcome measure HRQL more pragmatically represents what patients say about how they feel and function in their daily lives as result of a disease or treatment and it generally includes, at a minimum, items about physical function and symptoms, psychological state, and social interaction [3,4,8].

When choosing an HRQL instrument, the researcher or clinician needs to ask "has an instrument been developed and tested for the purpose for which it is to be used and does it meet my needs?" Asking this question can give direction to reading the literature and will help the researcher or clinician select the instrument best suited for the specific purpose. There are a number of heart disease specific HRQL instruments. Most are designed for a particular heart disease diagnosis, for example, the Seattle Angina Questionnaire [9], the Minnesota Living with Heart Failure questionnaire [10], the Kansas City Cardiomyopathy Questionnaire [11], and the recently published Myocardial Infarction Dimensional Assessment Scale [12].

For HRQL instruments to be useful, not only must they demonstrate acceptable validity criteria, they must also exhibit good discriminative, evaluative, and predictive properties [13]. A discriminative instrument is designed to distinguish between individuals or groups at a single point in time, e.g., good and poor HRQL, and needs to demonstrate reliability. An evaluative instrument is used when one needs to measure change in HRQL over time, e.g., as a result of an intervention, and needs to be responsive. A predictive instrument classifies individuals into a set of predefined measurement categories, e.g., alive or dead. Another key property of both discriminative and evaluative instruments is interpretability, i.e., can the differences between subjects at a single point in time be identified by discriminative instrument as trivial, small, moderate, or large and can the changes within subjects over time be identified by evaluative instruments as trivial, small, moderate, or large?

In this article, we review evidence concerning the measurement properties of the MacNew Heart Disease Health-related Quality of Life [MacNew] Questionnaire [14,15]. This instrument is a modification of the Quality of Life after Myocardial Infarction [QLMI] Questionnaire which was originally developed nearly 20 years ago for patients who had survived an acute myocardial infraction [MI] and who were referred for subsequent cardiac rehabilitation [16].

The Original Quality of Life after Myocardial Infarction [QLMI] Questionnaire

The original interviewer-administered QLMI was developed for use in a trial of cardiac rehabilitation in moderately anxious and/or depressed patients with a documented MI [16]. It was designed to assess a patient's feelings about a range of issues and concerns identified by individuals who have suffered an acute MI and includes items addressing physical [and somatic], psychological, and social domains. The QLMI was designed to evaluate how daily activities, as well as physical, emotional, and social functioning are affected by the disease and its treatment for the disease.

The original QLMI items were generated through interviews with physicians, nurses, allied health professionals, patients with MI, and by reviewing the literature [17]. The item reduction instrument consisted of 97 items. It was administered to a group of 63 patients who were recruited from a cardiac rehabilitation program, 3 community hospitals, and volunteer patients from the community. There were 57 males and 6 females (mean age 58.0 years) who had recovered from an acute MI, 59% within 3 months, 25% 3 to 6 months, and 6% >12 months previously. Patients were asked to identify items of concern to them, and to rate those items from 1 [not very important] to 5 [very important]. A clinical impact score was generated by multiplying the number of patients rating an item as of concern by the mean score of that item [18]. The items with the highest scores were selected to form the interviewer-administered QLMI Questionnaire. The items of the original QLMI instrument were conceptually assigned to five domains [symptoms, restrictions, confidence, self-esteem and emotion] [17] without including factor analytical strategies to confirm the theoretically assumed five-factor structure. The internal consistency of the original QLMI domains ranges from 0.75 to 0.87 [17]. A priori hypotheses to determine the discriminative and evaluative validity of the items in the QLMI were tested against several well established clinical measures [e.g. State-Trait Anxiety Inventory, Beck Depression Inventory, Profile of Mood Scale, Time Trade-off, Quality of Well-Being, Katz social functioning instrument, and a symptom limited exercise test]. Twenty-eight out of 34 hypotheses were accepted confirming the discriminative construct validity of the items [17]. Although the evaluative validity of the QLMI was not as robust as its discriminative validity [17], the responsiveness of the instrument has since been demonstrated in a number of studies [see the section on responsiveness below].

The MacNew Heart Disease Health-related Quality of Life instrument

The MacNew Heart Disease HRQL questionnaire [MacNew] is a self-administered modification of the original QLMI instrument [14,15]. The MacNew consists of 27 items which fall into three domains [a 13-item physical limitations domain scale, a 14-item emotional function domain scale, and a 13-item social function domain scale]. There are 5 items that inquire about symptoms: angina/chest pain, shortness of breath, fatigue, dizziness, and aching legs. The time frame for the MacNew is the previous two weeks.

Scoring of the MacNew is straight-forward. The maximum possible score in any domain is 7 [high HRQL] and the minimum is 1 [poor HRQL]. Missing responses do not contribute to the score and item 27, 'sexual intercourse', may be excluded without altering the domain score as each domain score is calculated as the average of the responses in that domain. For example, if only 10 of the 14 Emotional items are answered, the Emotional Score is the average of 10 responses. If more than 50% of the items for a domain are missing, the score for that domain is not calculated, that is, it is considered to be missing. The instrument also has a global HRQL score which can be calculated as the average over all scored items unless one of the domains is completely missing [14,15]. In addition to the 11 studies reported below which provide psychometric data [14,15,17,19-26], the MacNew has been successfully administered, to our knowledge, in at least 12 clinical and/or experimental studies [16,27-37] to more than 5,200 patients with heart disease

Psychometric properties

A disease specific HRQL instrument is one step towards individualizing the measurement of a patient's quality of life [38]. As either a discriminative or evaluative instrument it should meet the basic criteria for acceptable psychometric properties which include validity, reliability, responsiveness, and interpretability [39].

Reliability

Test reliability is the degree to which the test is free from random error [39]. The assessment of reliability includes two main forms: internal consistency and reproducibility. Internal consistency is typically assessed by Cronbach's alpha, while reproducibility can be assessed by test-retest and intraclass correlation coefficients [39]. Table 1 presents the available data on the internal consistency and reproducibility of the QLMI and the Dutch, English, Farsi, German, and Spanish versions of the MacNew.

Table 1

Internal consistency, reproducibility and responsiveness statistics for the English [n = 352], Dutch [n = 339], Farsi [n = 51], German [n = 357], and Spanish [n = 143] versions of the MacNew Heart Disease Health-related Quality of Life Questionnaire

	English	Dutch	Farsi	German	Spanish
Cronbach's α
Physical	.93	.86–.91	.92	.86 – .90	.88
Emotional	.95	.91–.92	.92	.97	.85
Social	.95	.78–.84	.94	.93 – .95	.83
Global	NA¶	.92–.95	.95	.93 – .97	NA¶
Test-Retest rtt
Physical	.83 & .75*	NA¶	.38	.68 – .87	.81
Emotional	.86**	NA¶	.42	.63 – .72	.78
Social	.87 & .85‡	NA¶	.31	.61 – .83	.71
Global	.86‡‡	NA.¶	.50	.65 – .81	NA¶
Intraclass Correlation
Physical	NA¶	NA¶	.92	.73 – .93	.87
Emotional	NA¶	NA¶	.92	.77 – .84	.83
Social	NA¶	NA¶	.93	.75 – .91	.83
Global	NA¶	NA¶	.95	.76 – .81	NA¶
Effect size
Physical	.27 & 1.34*	1.38	NA¶	.86	NA¶
Emotional	.57**	.81	NA¶	.94	NA¶
Social	1.43 & .85‡	1.11	NA¶	.99	NA¶
Global	1.22‡‡	1.18	NA¶	1.12	NA¶

¶ Not available * Symptoms and restrictions scales for QLMI at 12 months [n = 201] [17] ** Emotions scale in QLMI at 12 months [n = 201] [17]‡ Confidence and self-esteem scales in QLMI at 12 months [n = 201] [17]‡‡ Global score in QLMI at 12 months [n = 201] [17]

Internal consistency for the three domains in the English version of the MacNew ranges from 0.93 to 0.95 [15] and ranges from 0.75 to 0.97 in the Dutch [De Gucht, manuscript under review], Farsi [24], German [23,26], and Spanish [21] versions of the MacNew [Table 1]. As discriminative instruments measure differences between subjects, the intraclass correlation is usually considered to be the better indicator for reproducibility as it accounts for a possible systematic difference of the replicated measurements [39]. The intraclass correlation coefficients range from .73 – .95 in the international versions [21,23,24,26] [Table 1]. Test-retest correlations of the MacNew indicate acceptable reproducibility on separate occasions in patients with myocardial infarction and angina with ranges from rtt = .61 – .87 in the English [17], Spanish [21] and German versions [23,26]. In the Farsi version, the test-retest correlations, although significant, are lower but this most likely because patients did not necessarily meet criteria for clinical stability [24] [Table 1].

Overall the MacNew is reliable and meets the reproducibility standard of 0.70 for group comparison which usually is the goal in clinical trials [39].

Validity

A test is valid when it measures what it purports to measure. Three forms of validity are commonly recognized: face and content validity, construct-related validity and criterion-related validity. The last, however, can be difficult to establish in the absence of a widely accepted criterion measure [gold standard] [39]. In addition, predictive validity is an additional useful form of validity for HRQL instruments.

Face and content validity

Face and content validity of the MacNew were established during the process of developing the MacNew [17].

Construct-related validity

Construct-related validity of the MacNew as a discriminative instrument was carried out initially on the English version MacNew, and subsequently on various language translations. Construct validity was assessed by examining the logical relations that should exist with other measures and by using the "known-group" comparison strategy [40]. Development of the MacNew incorporated psychometric testing using principal component factor analytical approaches identifying three core underlying factors: physical, emotional and social [14,15]. These three domains explain between 63.0% and 66.5% of the variance [14,15]. Extensive substantiation of the logical relationships between the items, defined a priori, further demonstrated the discriminative construct validity of the MacNew [14,15]. For example, Lim et al [14] used gender, previous MI, coronary artery bypass surgery, readmission to hospital, positive attitude to exercise and dietary habits, and perceived threat of further heart problems while Valenti and colleagues [15] used age, gender, previous MI, coronary artery bypass surgery, and readmission to hospital as the discriminating factors.

Principal component factor analytical approaches have substantially confirmed the three factor (physical, emotional and social) structure of the original MacNew model [14,15] in each of the independently generated Dutch [items #24, 25, 26 were omitted from this version] [De Gucht, manuscript under review], Farsi [24], and Spanish [21] translations of the MacNew. Structural equation modeling used with the German version for MI patients also substantially confirmed the three factor structure [26]. In the original factor analyses [14,15], the decision was made to include any item with a factor weight of =>0.40 in that domain which meant that any one item could fall into more than one domain. Of the 26 items, five items were allocated to the emotional and social domains, five to the physical and social domains, one to the emotional and physical domains, one to all three domains, and the remaining 14 to one domain. The validity of this approach may be questioned and is presently being examined.

There apparently is relatively little impact of the different languages on the international "harmonization" of the original three factor structure. The highest factor weight for each item in each language is identified in Table 2 with the checks identifying the domain to which the item was allocated the original English factor analysis [15]. Of the 26 items, 25 [96%] matched the domain with the highest weighting in the original English analysis in at least two of the four languages and 15 [58%] items matched in at least three of the four languages; one item, #6, originally allocated to the emotional domain, matched in only one of the other four languages."

Table 2

Items [with the highest factor weightings =>0.40] as allocated to each domain in the original English version of the MacNew [] and the Dutch [D], Farsi [F], German [G], and Spanish [S] versions of the MacNew

Item	Emotional		Physical		Social
1. Frustrated		D, G, S				F
2. Worthless		D, G				F, S
3. Confident		D, G, S		F
4. Down in the dumps		D, G, S				F
5. Relaxed		D, F, G, S
6. Worn Out		S		D, G		F
7. Happy with Personal Life		D, F, G, S
8. Restless		D, F, G, S
9. Short of Breath				D, F, G, S
10. Tearful		D, G				F, S
11. More Dependent						D, F, S
12. Social Activities						D, F, G, S
13. Others/less Confidence in you		G				D, F, S
14. Chest Pain				D, G, S		F
15. Lack Self-Confidence		D, G				F, S
16. Aching Legs				D, F, G, S
17. Sports/Exercise Limited				D, S		F, G
18. Frightened		D, G		F		S
19. Dizzy/Lightheaded		S		D, G
20. Restricted or Limited				D, S		F, G
21. Unsure about Exercise				D, S		F, G
22. Overprotective Family						D, G, S
23. Burden on Others						D, F, G, S
24. Excluded				S		F, G
25. Unable to Socialize						F, G, S
26. Physically Restricted				S		F, G

These independent international studies also used other approaches to demonstrate discriminative construct validity and found significant correlations with the relevant concepts of each of the different instruments used. In the Dutch study the authors used the Heart Patient's Psychological Questionnaire and the Symptom Checklist to demonstrate construct validity [De Gucht, manuscript under review]. The SF-36 was used to confirm convergent validity in Austrian samples of angina patients and MI-patients [23,26]. The "known-group" approach, using established clinical indicators, was carried out in MI and angina patients as an additional way to establish validity. Patients were grouped according to clinical measures of disease severity, for example by the Canadian Cardiovascular Society and the New York Heart Association classification scales [23,25] or by either previous myocardial infarction, rehospitalization, or surgery [14,15]. MacNew domain scores of the groups differed in the directions expected, confirming the discriminative properties of the MacNew.

Predictive validity

The predictive validity of the MacNew has been examined by Lim and colleagues [19]. They tested the hypothesis that low HRQL after discharge from hospital with ischemic heart disease is associated with higher rates of adverse events [death or re-hospitalization] in 375 patients. The median [and interquartile ranges] Global domain scores were 4.1 [3.6–4.6] for the low HRQL group, 5.6 [5.3–5.9] for the moderate HRQL group, and 6.5 [6.2–6.7] for the high HRQL group and each of these differences exceeded the minimal important difference for the MacNew [see the interpretability section below]. An odds ratio of 2.66 [95% confidence limits = 1.2 to 5.8] for an adverse event in the next 24 months was observed in patients in the lowest tertile for global MacNew HRQL at discharge compared to those in the highest tertile with an absolute adverse event rate of 28% in the low HRQL and 9% in the high HRQL group [19].

Overall these findings support the face and content, construct, and predictive validity of the MacNew for evaluating HRQL in patients with heart disease and can be recommended for international use.

Responsiveness

A test is responsive when it is sensitive to change. Responsiveness is an important psychometric property of an HRQL instrument when the purpose is to evaluate treatment effects over time, especially if they are small. It is usually assessed with an effect size statistic [39]. The evaluative [longitudinal] validity, i.e., responsiveness, of the MacNew has been tested with three different effect size statistics, the effect size statistic [ES], the standardized response mean [SRM], and the responsiveness statistic [RS]. The denominator in each case is the difference between groups with the standard deviation at baseline as the numerator for the ES, the standard deviation of change for the SRM, and the standard deviation in stable patients for the RS.

As shown in Table 1, the SRM effect sizes over the longer 12-month follow-up period were moderate to strong [SRM ranging from 0.57 to 1.43] for five of the six scales [17]. The German version of the MacNew was administered to angina patients at the time of angiography and again at the 12-month follow-up following continued medical or invasive treatment methods [23]. In this study, the responsiveness of the German MacNew over the 12-months is moderate using the ES and the SRM [0.30 to 0.47] but strong with the RS [0.86 to 1.12] [Table 1]. By treatment, the ES was <0.27 with continued medical treatment, ranged from 0.39 to 0.51 with percutaneous coronary intervention, and from 0.60 to 0.70 with coronary bypass surgery confirming the expected impact of different interventions [23]. Limited data on the MacNew suggest that HRQL is significantly improved in patients with heart failure although no attempt was made to calculate an effect size [36].

There are some comparative data for the responsiveness of the MacNew and other HRQL instruments. For example, using the RS analysis, the effect size for the MacNew physical domain, which measures a patient's perception of the physical function, is 0.86 compared to 0.59 for the Seattle Angina Questionnaire physical limitations domain which measures a patient's physical performance [23]. On the other hand, if the Seattle Angina disease perception and the MacNew Global scores can be compared, the Seattle disease perception domain RS is 1.48 and higher than the RS observed for the German Global MacNew score at 1.12 [23]. In the Dutch study, the MacNew was considerably more responsive [ES, 0.81 to 1.38] than either the Symptom Checklist [ES, 0.11 to 0.67] or the Heart Patient's Psychological Questionnaire [ES, 0.06 to 0.28] [De Gucht, manuscript under review]. Further, there is additional experimental evidence for the responsiveness of the QLMI and MacNew as an outcome measure in cardiac rehabilitation [28,32,37].

Overall, these observations suggest that, as an evaluative instrument, the MacNew is responsive and sensitive to changes in HRQL following various interventions for patients with coronary heart disease.

Interpretability

Interpretability is concerned with the understanding of the meaning of the instrument's quantitative scores, especially for change scores [39]. The minimal important difference is defined as "the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management" [41]. Dixon et al. [22] investigated the minimal important difference for the MacNew domain scores, hypothesizing that scores of successfully revascularized patients would increase, that scores of readmitted patients would decrease, and that the scores of the remainder would be relatively stable. They were able to demonstrate that a change of at least 0.5 is a useful indicator of the minimal important difference for all 3 scales and the global score [22]. This observation has since been substantiated in other work [25,36,42]. Dixon and colleagues [22] have provided the user of the English-version MacNew with normative data which are available by diagnosis [myocardial infarction, angina, and heart failure] and age [<65, 65–74, and 75–85 years].

Respondent and administrative burden

The MacNew has a low respondent-burden [14,15,23,26] taking on average 10 minutes or less to complete, and its acceptability is demonstrated by response rates of over 90%. With the availability of SPSS-Syntax to score the MacNew, there is also little researcher-burden.

Availability and Cost of the MacNew

The MacNew is copyrighted by the developers for the purpose of maintaining an unmodified version in order to preserve the integrity of the instrument. As the MacNew is considered to be in the public domain, there is no cost for permission to use the MacNew or its translations. However, there is a request that investigators use the relevant citations in their publications and provide the developers of the MacNew with published data to further develop the instrument.

The original English version of the MacNew and its translations into Chinese [Cantonese], Danish, Dutch, Farsi, Finnish, French, German, Greek, Hebrew, Italian, Lithuanian, Norwegian, Polish, Portuguese, Romanian, Russian, Spanish, Swedish and Turkish are available from the developers of the MacNew. The basic descriptive information on the MacNew is available on the QOLID website at .

Conclusion

HRQL provides researchers and clinicians with valuable additional information about the impact of either the condition or treatment from the patient's perspective, particularly in chronic diseases such as coronary heart disease which is a major cause of disability. We believe that the MacNew Heart Disease Health-related Quality of Life instrument meets the established criteria for the psychometric properties of instrument reliability, validity, and responsiveness for use in patients with myocardial infarction and in patients with angina. The English and the German MacNew are presently being tested, with considerable preliminary promise, in patients with heart failure. Psychometric assessment of the German MacNew in pacemaker patients is nearing completion. Further, tests of the psychometric properties of eleven of the MacNew translations are underway in patients with myocardial infarction, with angina, and with heart failure in an international study which is being coordinated through the Working Group on Cardiac Rehabilitation and Exercise Physiology of the European Society of Cardiology.

Contact to obtain the MacNew

The MacNew package containing a brief description of the instrument, citations, the scoring algorithm, and the MacNew questionnaire can be obtained from the following:

Neil Oldridge, PhD

Center for Urban Population Health

University of Wisconsin-Milwaukee

1020 N. 12th Street, OHC 4th Floor

Milwaukee, WI 53201-0342

Telephone: 414-219-4084

Fax: 414-219-6563

Email neilb@uwm.edu

Author contributions

GG and NBO developed and validated the original QLMI; LL modified the QLMI as the MacNew and validated it; SH validated the German MacNew and prepared the initial draft of this manuscript; SH, LL, GG, and NBO jointly edited the draft manuscript. All authors have read and approve the manuscript.