Economic analysis of the health impacts of housing improvement studies: a systematic review.

BACKGROUND: Economic evaluation of public policies has been advocated but rarely performed. Studies from a systematic review of the health impacts of housing improvement included data on costs and some economic analysis. Examination of these data provides an opportunity to explore the difficulties and the potential for economic evaluation of housing.
METHODS: Data were extracted from all studies included in the systematic review of housing improvement which had reported costs and economic analysis (n=29/45). The reported data were assessed for their suitability to economic evaluation. Where an economic analysis was reported the analysis was described according to pre-set definitions of various types of economic analysis used in the field of health economics.
RESULTS: 25 studies reported cost data on the intervention and/or benefits to the recipients. Of these, 11 studies reported data which was considered amenable to economic evaluation. A further four studies reported conducting an economic evaluation. Three of these studies presented a hybrid 'balance sheet' approach and indicated a net economic benefit associated with the intervention. One cost-effectiveness evaluation was identified but the data were unclearly reported; the cost-effectiveness plane suggested that the intervention was more costly and less effective than the status quo.
CONCLUSIONS: Future studies planning an economic evaluation need to (i) make best use of available data and (ii) ensure that all relevant data are collected. To facilitate this, economic evaluations should be planned alongside the intervention with input from health economists from the outset of the study. When undertaken appropriately, economic evaluation provides the potential to make significant contributions to housing policy.

Introduction

Economic evaluation of health technologies (drugs, devices, etc) has become widespread across the world, with government agencies (eg, National Institute of Clinical Excellence (NICE) in England and Wales) employing these techniques on a routine basis to make decisions about which healthcare to fund.1–4 But there has been very little progress in economic evaluation for public health and in particular within public policy.5–8 In 2005, the remit for NICE was expanded to include public health, reflecting a growing desire to broaden the scope of such evaluations and address issues of resource allocation across all sectors impacting on health.9 10

Provision of acceptable housing conditions may be regarded as a cornerstone of healthy public policy, representing a major public investment with the potential to improve health and contribute wider public health strategies to improve population health and reduce health inequalities.11–13 In 2009, a systematic review of the health impact of housing interventions, including studies from around the world, concluded that housing improvements can lead to health improvements.14 This was especially the case for warmth improvements targeted at those with poor health, living in poor housing. Some of the studies in the 2009 systematic review included reports on the costs associated with the interventions and a small number reported having undertaken economic evaluations.14 Given the growing desire of policy makers to demonstrate value for money from interventions, we undertook a further review of these studies to identify and extract data which could be used to inform estimates of the relative costs and benefits of housing improvement, and illustrate the challenges of economic evaluation in this field.

This paper presents the results of this review, providing details of the cost and economic analyses reported alongside housing intervention studies and accompanying health impacts. The paper uses these data to examine the economics of housing investment and also to reflect on the current state of health economic analysis in housing. As an example of health economic evaluation of a substantial public investment and policy area, the lessons may have a wider methodological relevance to topics of interest to healthy public policy, such as welfare reforms or transport initiatives.

Methods

Prior to detailing the process undertaken for this systematic review of economic data and analysis, we detail the definitions used to distinguish between differing types of cost studies and economic evaluations (table 1). These definitions are routinely applied in the field of health economics and have been proposed for the economic appraisal of public health interventions.4 5 7 15

Table 1

Types of economic studies (an assessment of cost is common to all study types detailed)5 7 15

Type of economic study	Description	Outcomes	Further explanation
Cost-offset study	Presents the costs (incurred and saved) associated with the policy change	Not measured	No consideration of health outcomes
Cost-minimisation analysis	Presents the costs (incurred and saved) associated with the policy change compared to the status quo	Assumed equal	Involves the assumption that the outcomes associated with the new policy are equivalent to those associated with the status quo, thus it is sufficient to compare the costs of the two to identify the least costly way of achieving the outcome. It must be noted that true equivalence of outcomes is rare and cost-minimisation analysis is often used inappropriately
Cost-consequences analysis	Presents a detailed listing of the various impacts on outcomes associated with the policy change with no attempt to value the aggregated components in a single metric	Range of outcomes listed	A useful first step towards economic evaluation. No attempt to combine or simplify outcomes into a single measure of effectiveness. Unless all of the individual outcome components move in the same direction, in order to determine whether a policy is worthwhile, it will be necessary to aggregate the various components and, with cost-consequences analyses, the burden for this will fall on the policy maker, who must decide the weightings associated with each component.
Cost-effectiveness analysis	Presents the outcomes associated with the policy change, and the status quo, in terms of uni-dimensional health or clinical units. The additional costs associated with the policy change are then presented in terms of a cost-effectiveness ratio, as, for example, the additional cost per additional asthma attack averted	Measured in health units (eg, asthma attacks, mental health score, physical health score)	Cost-effectiveness analysis requires that there is a single measure of outcome that captures the impacts of the policy. The main issue for cost-effectiveness analysis is that it can only be used to compare policies that generate the same outcomes. For example, it would not be possible to compare the cost-effectiveness of a housing intervention for which the outcome is measured as asthma attacks averted, with an intervention for which the outcome is measured as change in mental health score
Cost-utility analysis	Special case of cost-effectiveness analysis where the outcomes are presented as utility values. The additional costs associated with the policy change are presented in terms of a cost-utility ratio (often referred to as the cost-effectiveness ratio), as, for example, the additional cost per QALY	Measured as utility score reflecting both the health outcomes and preferences for them (eg, QALY)	Cost-utility analyses can be used to compare policy changes in different areas, with different natural outcomes, by providing a common measure of outcome. The most commonly used measure of utility is the QALY which incorporates measures of the quantity of life with assessments of the quality of life. The main issue for cost-utility analysis is that it may not capture the broader non-heath consequences associated with the policy
Cost-benefit analysis	Presents the outcomes associated with the policy change, and the status quo, in monetary units. The monetary value of the outcomes is then simply compared to the costs; any policy change where the monetary value of the outcomes outweighs the costs is therefore considered worthwhile	Measured in monetary terms	Cost- benefit analysis can be used to compare policy changes in different areas. Outcomes not limited to health consequences, will include all outcomes associated with the policy that are of importance to the individual. It is important to note that the term ‘cost-benefit analysis‘ is frequently misused to represent economic evaluation in general or cost-offset, cost-minimisation or cost-effectiveness analysis

QALY, quality adjusted life year.

Cost studies

Table 1 details three forms of cost study which are common within the literature, namely cost-offset, cost-minimisation and cost-consequence. Cost-offset studies simply detail the costs of the intervention alongside the cost savings achievable (eg, days in hospital averted). There is no measurement of health outcome. Cost-minimisation studies compare the costs (including any cost savings) for an intervention with the costs of the status quo under the assumption that the outcomes are equivalent between the two. Cost-consequence studies present the costs (including any cost savings) associated with an intervention and the status quo alongside a list of the various possible outcomes achieved. There is no attempt to identify or value the collective outcomes within a single metric. As such, while cost consequence studies provide a useful descriptive summary, and first step towards a full economic evaluation, they cannot be used to determine value for money or identify priority interventions.

These approaches are not formally economic evaluation techniques because they do not allow for a formal comparative analysis in terms of costs and outcomes. In a situation where policy involves the provision of a specific amenity (eg, a new heating/insulation system) then a comparison of costs is all that is required to establish the least costly way to provide the desired amenity. However, this assumes that provision of the specific amenity is, by some definition, a good thing (eg, it improves health outcomes).

Economic evaluations

Where budgets are constrained there is an unavoidable opportunity cost of undertaking any policy, as funds spent on one intervention will limit the funds available for other interventions. The aim of economic evaluation is to assist policy makers to identify interventions/policies which represent good value for money. The value for money associated with a new intervention is determined by comparing the additional costs required and additional outcomes achieved, with the status quo or current intervention. Where an intervention/policy provides greater outcomes at lower costs, it is said to ‘dominate’ the current intervention/policy and a decision about adopting the new intervention/policy is straightforward. However, where an intervention/policy provides greater outcomes at greater cost, a decision must be made about whether to spend these additional resources to achieve these additional outcomes. Where outcomes of different interventions/policies are valued using common metrics (see table 1), the value for money associated with them can be directly compared to indicate where these outcomes can be achieved at the lowest price. These interventions represent the best value for money compared with the alternatives available and, where resources are scarce, could be considered ‘economically worthwhile’.

There are three main forms of economic evaluation: cost-effectiveness analysis, cost-utility analysis and cost-benefit analysis (table 1). Each method involves a ‘comparative analysis of alternative courses of action in terms of both their costs and consequences’.4 With each method costs are measured in monetary units, while the measurement and valuation of outcomes differs. As such, the methods are traditionally classified by outcome.15

Cost-effectiveness analysis typically involves measuring a specific, one-dimensional, health or clinical outcome, for example asthma attacks averted. Cost-utility analysis is a special type of cost-effectiveness where multidimensional health outcomes are reduced to a single dimension reflecting individuals’ preferences for the diverse health outcomes. The most commonly used outcome in cost-utility analysis is the quality adjusted life year (QALY). For both cost-effectiveness and cost-utility studies, value for money is identified using a measure of the additional cost per additional outcome ratio (eg, an incremental cost/QALY ratio) and comparing that to an external threshold or to the ratio achieved by alternative policies. In contrast, cost-benefit analysis involves the measurement and valuation of all outcomes of interest in monetary terms. Here the value for money is identified by positive net economic benefit associated with the interventions (ie, the monetary value of the outcomes exceeds the net costs of the intervention less any costs savings achieved elsewhere). This Paretian definition of cost-benefit analysis is the established, standard definition used within health economics. It allows a broad spectrum of outcomes (all those of importance to the individual) to be included within the metric, but requires the, often complex, valuation of outcomes in monetary terms.

Data extraction and analysis of economic data in housing intervention studies

All housing intervention studies and their associated papers included in the 2009 systematic review of the health impacts of housing improvement14 were examined for reports of costs and economic analyses. Details of the scope of the review (inclusion and exclusion criteria), and evidence appraisal are available in the 2009 publication along with the findings of the review.14 Forty-five medical and social science databases, as well as websites and grey literature were searched to identify studies of housing improvement which assessed change in any health outcome. A separate search for economic studies was not undertaken but economic studies which included health outcomes following housing improvement would have been identified in the broad search. Two independent reviewers screened 27 082 citations to select the included studies. All available data on costs and, where available, details of any economic analysis were extracted by one reviewer (CM) and checked by a second reviewer (EF or HT). The cost data were tabulated alongside a summary of reported heath impacts and an indication of overall study quality as used in the original review (A=minimal bias, B=some bias, C=considerable bias).14 Where a study reported plans to undertake economic analysis, the authors were contacted for an update on progress and available data, or reasons for not completing the economic analysis.

Studies were allocated into two groups based on the type of data reported. Studies which only presented cost data (table 2) were further examined for the potential to have conducted an economic evaluation, that is, presence of a suitable health outcome which could be linked to cost and compared to an alternative (table 1). Studies which reported having undertaken an economic evaluation (table 3) were examined to determine the precise form of that analysis (table 1).

Table 2

Summary of housing improvement studies reporting cost data without economic evaluation (ordered by intervention type, study quality and date)

Author, date, location	Study quality	Study design	Intervention costs	Costs to recipient	Potential for economic evaluation	Summary of economic data and accompanying authors’ interpretation
Intervention: Warmth and energy efficiency improvements (post-1980)
Howden-Chapman et al, 2008, New Zealand16	A	RCT	✓		Yes	Mean cost of intervention per house $(NZ)3000.
Platt et al, 2007, UK18	A	Contr'd B&A		✓	No	Intervention group in receipt of improve heating were less likely to report difficulties to ‘manage financially’, than those who did not acquire heating (OR: 0.77, 95% CI: 0.60 to 0.99).
Heyman et al, 2011, UK47 56 57	B	RCT	✓	✓	Yes: SF-36 amenable to QALY calculation	Mean cost of intervention per house £727. Mean fuel expenditure (Int/Cont n=99/83) £596/£567, p=0.408. Change mean warmth satisfaction score (4 point scale) Int/Cont (n∼96/82) +1.18/+0.64
Lloyd et al, 2008, UK17	B	Contr'd B&A		✓	No	Heating costs per week Before vs After (>4 years after intervention) (Int/Cont n=75/40) £35 vs £7 per week, no change in rent. Control group do not report any changes in housing costs (unclear how data were obtained).
Shortt et al, 2007, UK19 58 59	B	Contr'd B&A		✓	No	Fuel costs per annum Before vs After (Int n=54) £1113 vs £751.56. (Data refers to sub-group who received full intervention, no data for changes in control group).
Warm Front Study Group, 2006, UK21 48 60–63	C	Contr'd B&A	✓	✓	Yes	Maximum value of grant per house £2500. Following introduction of cavity wall and loft insulation space heating fuel consumption reduced by 10% in centrally heated properties and 17% in non-centrally heated properties. Gas central heating system did not change fuel consumption due to increased internal temperature.
Allen, 2005, UK22 49	C	UBA	✓		Yes	Mean cost of intervention per house £4477 (range £799–£10 144), total cost of project £176 297. Total funding from various grants (mean award per eligible property): disabled facilities £66 173 (£5494); occupational therapy £1691 (£85); renovation grant £14 081; home repair grant £8811.
Allen, 2005, UK23	C	UBA	✓		Yes	Mean cost of intervention per house £5800 (range £350 to £14 000).
Eick et al, 2004, UK24	C	RCT	✓	✓	No	Change in health costs in 3 months since intervention (n=16): GP visit −£136; GP home visit −£22.58; outpatient visit −£220; hospital admission −£5740; steroids −£5.70; antibiotics −£16.80; nebuliser −£1.23; absence from school due to asthma –£478.42; absence from school due to other reasons +£62.95Expenditure associated with the installation of MVHR system: cost of unit and installation £2500; annual maintenance by owner £150; annual running costs for occupant £35.
Caldwell et al, 2001, UK25	C	Contr'd B&A		✓	No	Changes in annual energy costs since intervention (participant recall) in four intervention sites −£235.46/−£457.70/−£206.47/−£254.60. No change reported in control sites
Green et al, 1999, UK26 64	C	RC	✓	✓	Yes	Mean cost of intervention per house £28 000. Estimated heating bills Int/Cont £4.46 vs £9.04 (difference in fuel costs attributed largely to differences in fuel supplier tariffs rather than consumption); weekly rent Int/Cont £29.64/£19.63
Intervention: Rehousing/retrofitting ± neighbourhood renewal (post 1995)
Thomson et al, 2007, UK27	A	Contr'd B&A		✓	No	Rent data presented for 33 (Int/Cont 18/15) participants. Mean rent per week at baseline Int/Cont £32.24/£31.00. Mean change in rent per week Int/Cont +£6.65/+£1.31. Some residents reported increased fuel cost (Int/Cont 14/5) actual cost data not presented.
Critchley et al, 2004, UK29	A	Contr'd B&A		✓	No	Over 12-year period Liverpool Housing Action Trust invested £260 m in housing renewal (this appears to be the total housing budget and does not refer specifically to this intervention). Estimated annual running costs Before vs After rehousing for: two person household £662 vs £347; single person's costs £610 vs £319. Percentage living in fuel poverty(excluding housing benefit), Before vs After intervention for- 1 person households 86% vs 14%; 2 person households 48% vs 8%.
Thomas et al, 2005, UK28 65	B	Contr'd B&A	✓		Yes	Total cost of renewal project (Single Regeneration Budget) £2 million over study period. Costs do not relate necessarily to study sample and not specifically to housing, project included range of non-housing investment improvements.
Blackman et al, 2001, UK30 66	C	UBA	✓		Yes	Mean cost of renewal project per house £8000. Total cost of housing renewal programme £5.5 million. Costs do not relate necessarily to study sample and not specifically to housing, project included some environmental improvements and road safety improvements.
Ambrose, 2000, UK31 32 67	C	UBA		✓	No	Before vs 2 years After intervention. Mean weekly housing costs: rent (n=105 households) £52 vs £72; water (n=60) £0.92 vs £4.38; gas (n=92) £5.54 vs £6.46; electricity (n=98) £4.62 vs £5.77. Before vs 3 years after intervention (n=19 households): mean weekly housing costs: rent £60.33 vs £79.30; water £3.50 vs £5.06; gas (n=9) £8.28 vs £6.15; electricity (n=6) £4.76 vs £3.33. (cost data collected retrospectively, not all participants reporting cost data also reported health data)
Walker and Bradshaw, 1999 UK33	C	XCBA	✓	✓	No	Investment of £8.6 million by local authority in repair of homes and renovation of property. Percentage change in general practice prescribing costs per 1000 patients after intervention (1994–1998). Intervention practice A/Intervention practice B/Control practices (n=7): gastrointestinal +12.33%/+25.8%/+12.92%; cardiovascular +31.27%/+37.56%/+27.01%; respiratory +46.92%/+82.87%/+43.57+; central nervous system +79.22%/+73.7%/+79.7%; hypnotic +67.58%/+15.99%/+93.33%; anxiolytic −74.12%/−12.29%/−6.51%; antidepressant +109.51%/+86.27%/+120.77%; analgesic +26.92%/+26.59%/+42.66%; anti-infective +12.96%/−22.19%/−26.26%.
Woodin et al, 1996, UK34	C	R	✓		No	Total cost of renewal project £97 million, figure includes more than study sample.
Halpern, 1995, UK35	C	XUBA	✓		Yes	Mean cost of intervention per house (full refurbishment) £10 000–£15 000.
Intervention: Provision of basic housing needs/low or middle income country
Cattaneo et al, 2006, Mexico36	B	RC	✓		Yes	Mean cost of intervention per house=$(US)150.
Aga Khan Health Service, 2001, Pakistan38	B	RC		✓	No	Annual spending (rupees) on health care after intervention (Int/Cont n=50/99) 0 rupees=14%/16%; 1–999 rupees 18%/18%; 1000–5999 rupees 26%/39%; >5999 rupees 26%/16%; don't know 16%/10%. Estimated that insulation has resulted in up to 50% reduction in wood consumption by a typical family, reducing cost and time spent collecting and buying firewood.
Aiga et al, 2002, Phillipines37 68	C	R		✓	No	Mean monthly household expenditure on water (Pesos) Int/Cont 109/234. Intervention cost not available. Mean household income (Pesos) after intervention Int/Cont 8032/4530. Increased household income attributed to increase in time available to earn (as a result of improved water supply). Estimated that increased income of 5740 Pesos in control group if they received improved water supply.
Wolff et al, 2001, Malawi39	C	UBA	✓		Yes	Mean cost of building a new ‘habitat’ house, $ (US) 550.
Intervention: Rehousing from slums (pre-1965)
McGonigle et al, 1936 UK41	B	Contr'd B&A		✓	No	Mean weekly rent Before vs After Int 4s.8d/9s.0d. Cont 4s.7¾d/4s.10¾d (s=shilling, d=pence, 1 shilling=5 pence). Rent as a % of income subdivided by employed or unemployed status (Int/Cont n=28/27 families). Int Employed/unemployed 20.5%/31.3%; Cont Employed/unemployed 14.7%/20.8% (Int/Cont n=35/30 families) Before vs After Int/Cont rent as % of income 20.5%/14.7% vs 31.3%/20.8%.
Chapin, 1938, USA40	C	UBA	✓	✓	No	Mean cost of intervention per house $(US) 7791 total cost of project $3 623 000 for 465 houses. Before vs After intervention, mean unit rental $(US) 15.68 vs 17.98, mean room rental $(US)3.21 vs 3.79.

*Study design: RCT, randomised controlled trial; Contr'd B&A, controlled before & after; UBA, uncontrolled before & after; XCBA, controlled before & after using area level cross sectional data at both time points; XUBA, uncontrolled before & after using area level cross sectional data at both time points; RC, retrospective controlled study; R, retrospective uncontrolled.

QALY, quality adjusted life year.

Table 3

Housing improvement studies reporting an economic evaluation (ordered by intervention type, study quality and date)

Author, date, location	Description of housing intervention	Study design	Summary of effect directions on health outcomes by domain**				Summary of economic data and analysis reported
			General health	Respiratory	Mental health	Illness/symptom
Intervention: Warmth and energy efficiency improvements (post-1980)
Barton et al, 2007, UK31 45 69	Upgrading heating provision and energy efficiency according to need. For some houses, roofs were fitted with breathable roofing felt, plus 50 mm insulation, Cavity insulation with rockwool fibres, and double glazing. Over ceiling insulation topped up to 200 mm (glass fibre quilting), Front and back doors and French windows were replaced with uPVC doors.	RCT	<>	◄►	<>	▾	Paper states cost effectiveness analysis was carried out using SF-36 data, and report no significant differences between groups or over time in SF-36 subscales, but no data reported. Bootstrapped cost-effectiveness estimate (presented graphically) suggests that intervention dominated (more costly, less effective).Costs of intervention for each year. 49 houses improved in 1999, cost per house of £7760, 63 houses improved in 2000, average cost per house of £4819
							Compares costs (£GBP) between intervention and control groups for 3 years
							Costs per person (Year 1999)	Intervention	Control
							Annual equivalent intervention costs net of annual energy saving	£0	£0
							All NHS costs*	£204.86	£220.49
							Benefits
							SF36 (adults)	0.73	0.75
							Value of lost education	£240.7	£288.44
							Costs per person (Year 2000)	Intervention	Control
							Annual equivalent intervention costs net of annual energy saving	£-18.82	£0
							All NHS costs*	£224.97	£171.54
							Benefits
							SF36 (adults)	0.77	0.8
							Value of lost education	£352.28	£247.59
							Costs per person (Year 2001)	Intervention	Control
							Annual equivalent intervention costs net of annual energy saving	£0	£-16.56
							All NHS costs*	£135.72	£165.56
							Benefits
							SF36 (adults)	0.72	0.8
							Value of lost education	£80.92	£169.92
Chapman et al, 2007, New Zealand42 50 70 71	Ceiling insulation, draught-proofing of windows and doors, sisalated paper (insulated foil) strapped under floor joists, and polyethylene covering over the ground.	RCT	▴	▴	▴		Cost of intervention per household was $(NZ)1800. Assumed lifetime of benefits is 30 years
							Economic value of benefits ($(NZ)).
							Change in GP visits	165*
							Reduced hospital admissions	2231
							Reduced days of school	242
							Reduced days of work	179
							Energy savings	786
							CO2 savings	100
							Total savings	3374
							* negative saving
							Calculated per household: savings $NZ 3374; intervention cost $NZ1800. Net benefit of $1574 per household. Benefit-cost ratio of 1.87:1 Estimated value of savings was around $NZ 25 a year.
Mackenzie et al, 2000, UK44 46	Grant up to £2500 to improve heating and reduce damp and mould growth in house, intervention agreed according to need. (Gas central heating, n=28 (47%), electric storage heater, n=22 (37%), solid fuel central heating, n=7 (12%), oil-fired central heating, n=2 (4%)).	UBA		▴			Average cost of intervention £3061 per houseData on health service contacts and prescribing data (before and after intervention) available for 47 children (48% 47/97 invited to take part)Costs estimated for NHS contacts and prescriptions and school absences.
							Annual equivalent cost of improvements	£329.49
							Less
							Estimated annual saving on fuel bills	214.81
							Est. annual saving on NHS treatment costs	£499.54
							Est. annual increase in prescribing costs	−£11.41
							Est. annual value of increased school att.	£108.36
							Total estimated benefits from home improve	£810.23
							Net benefit per annum	£413.33
Laing and Baker, 2006, New Zealand43 72–74	Insulation (26.5%) & ventilation (43.5%) improvements, improved heating system (4.4%), extensions (8.7%), plus housing and health advice, improved links with health and other support agencies	RC	No health impacts only service use				In house cost benefit analysisDirect benefit to cost ratio −0.87 net present value to household, $(AUS)2222; when some indirect benefits included benefit to cost ratio 1.15 and net present value to household of $(AUS)2471No detail of methods or outcomes measures for analysis are provided. CBA planned

*Study design: RCT, randomised controlled trial; CBA, controlled before & after; UBA, uncontrolled before & after; RC, retrospective controlled study.

**Effect direction: upward arrow, positive health impact; downward arrow, negative health impact; sideways arrow, no change/mixed effects/conflicting findings.

Sample size: final sample size (individuals) in intervention group: large arrow, >300; medium arrow, 50–300; small arrow, <50.

Statistical significance: black arrow, p<0.05; grey arrow, p>0.05; empty arrow, no statistics/data reported.

Results

Forty-five studies were identified in the original review. The study designs varied and included five randomised controlled trials (RCTs), and 23 non-randomised controlled studies. The better quality RCTs and controlled studies were used to draw conclusions about effectiveness. The health outcomes reported included validated measures, for example SF-36, and self-reported measures, and covered four main domains: general health; mental health; respiratory health; and other/illness and symptoms. Twenty-nine studies reported costs or an economic analysis; 25 of these studies16–41 presented only basic cost data (table 2) while four studies reported having undertaken an economic analysis (table 3).42–46

Studies which present cost data without economic evaluation

Details of the 25 studies reporting cost data without economic analysis are presented in table 2 (see supplementary table 1 for full details of interventions, economic data and health impacts). Eleven of the studies examined warmth and energy efficiency interventions,16–26 eight examined rehousing or retrofitting,27–35 two focused on pre-1965 rehousing from slums40 41 and four focused on provision of basic housing needs.36–39 Over 75% of the studies (n=19) were from the UK,17–35 41 with one study each from New Zealand,16 the USA,40 Mexico,36 Philippines,37 Pakistan38 and Malawi.39 The study design and methodological quality of the studies varied, as assessed by the original systematic review criteria. These studies reported a diverse range of health outcomes, and the reported impacts suggest either improvement or no change in health status following housing improvement during the study period (see supplementary table 1). One study reported a deteriorated health status following housing improvement.22

Six studies24 26 33 40 47 48 presented data on both the cost of providing the intervention and other costs, including those incurred by the recipient, nine16 23 28 30 34–36 39 49 presented only costs relating to the provision of the intervention and ten studies18–19 25 27 29 31 37 38 41 presented only other costs, primarily to the recipient. The cost measures used varied and require different interpretations. For example, some studies reported intervention costs of a major housing led regeneration programme for an area beyond the included study population, while other studies reported average costs of the specific housing intervention per household. Measures used for recipient costs did not always report direct data and some studies used residents’ assessments of these changes, for example changes in fuel consumption or bills, making it difficult to interpret. The measures of other costs to recipients were reported, nine studies reported changes in fuel use or costs,17 19 25–27 31 38 47 48 six changes in rent,17 26 27 31 40 41 three changes in healthcare spending,24 33 38 two changes in household costs17 29 and one changes in income.37 One study compared the ability to manage financially between the intervention and control group.18 Drawing on the most commonly reported recipient costs, all six studies which reported numerical data for changes in fuel costs reported a reduction following energy efficiency improvements.17 19 25 26 31 47 Each of the five studies which reported numerical data for changes in rent reported increased rent following housing improvement.26 27 31 40 41

Of the 25 studies which presented only cost data, 11 (8 from the UK) were assessed to have sufficient data for an economic evaluation. Analysis linking cost data to health outcomes could have been conducted to present a cost-effectiveness, cost-utility or cost-benefit analysis (table 2).16 20–23 26 28 30 35 36 39

Two studies, both from the UK, reported plans to conduct an economic evaluation but this was either not conducted or not publicly available at the time of this review. Neither of these studies was among the 11 reporting data amenable to economic evaluation; Eick et al24 did not have data for a comparator group and Caldwell et al25 did not have data on intervention costs. Eick et al24 presented data on medical costs before and after the intervention and reported plans for a cost-benefit analysis. Caldwell et al (2001) originally planned to examine issues of cost-effectiveness but this was not undertaken due to poor quality data.

Studies reporting an economic evaluation

Four studies reported undertaking an economic evaluation (table 3). These studies all involved warmth and energy efficiency interventions since 2000 or later. Two studies were from the UK44–46 and two from New Zealand.42 43 50 The methodological study quality, with respect to assessment of health impacts in the original systematic review, varied; two studies were assessed to have a minimal risk of bias (grade A),42 45 50 one study was assessed to have some risk of bias (grade B)44 46 and one study was assessed to have considerable potential for bias (grade C).43

Three of these studies reported undertaking a cost-benefit analysis. These studies fall short of full cost-benefit analysis, as defined above, as they did not include a monetary valuation of all important outcomes. They are more accurately described as having presented a ‘balance sheet’ type approach. This hybrid approach involves identifying and listing the costs and benefits associated with an intervention or policy change, in much the same way as in a cost-consequence study.7 51 52 The costs and some of the benefits are then measured in monetary units where appropriate values are either available or can be postulated, otherwise they are simply listed in their natural units (eg, time). For example, a UK study44 presented cost data for medical treatments, prescriptions and fuel use and imputed a monetary benefit of reduced school absences. A study from New Zealand42 50 presented a benefit-cost ratio based on the cost of the intervention, changes in the costs of medical service use, and the economic value imputed for reduced CO2 emissions and the reduction in lost days of school and work. Another study from New Zealand43 presented a feasibility study for a cost-benefit analysis with a direct benefit to cost ratio, however the authors provided no details of the methods used or the outcomes measured. The findings from all three studies suggested net economic benefits associated with the interventions based on the outcomes measured in monetary terms (ie, the monetary value imputed for these outcomes exceeds the net costs of the intervention less any costs savings achieved elsewhere). In addition, both New Zealand studies indicate small, but positive, benefits to cost ratios associated with the intervention.42 43 50

One study reported undertaking a cost-effectiveness analysis which met the criteria employed by this review.45 This UK study used health indicator data (SF-36) to conduct a cost-effectiveness analysis. Although data were not presented numerically (it was narratively reported that the results were not statistically significant), a bootstrapped cost-effectiveness estimate presented graphically suggested that the intervention was dominated. This means that the intervention (improved housing) was more costly and less effective than the status quo.

Additional studies

Two further economic studies of housing interventions were identified when contacting study authors about completion of ongoing housing studies which might contribute to an update of the original housing review. These studies were published recently and were not included in the original systematic review.53 54 Edwards et al53 report an economic evaluation, as defined here, presenting an estimate of the additional cost per point improvement on the PedsQL asthma specific scale. Grimes et al54 present a ‘balance sheet’ approach with a calculation of net economic benefit.

In addition, we are aware of the economic evaluation undertaken alongside the Scottish Housing and Regeneration Project (submitted to JECH Lawson, Kearns, Petticrew, Fenwick, Investing in health: is social housing value for money? A cost-utility analysis.); however at the point at which our review was undertaken, results for this analysis were not available.

Discussion

Studies investigating the health impacts of housing improvement have frequently provided some details on costs or economic analysis (n=29/45).14 However, the majority of these (n=25/29) present data on intervention and/or recipient costs only and, despite sufficient data, opportunities to conduct economic analysis have been missed. Where studies report conducting economic evaluations, the majority of the reported analyses would be more accurately described as a ‘balance sheet’ approach.

Findings from the studies which report costs only, suggest that fuel costs may reduce following provision of warmth and energy efficiency improvements, and that rents may increase following housing improvement. These findings need careful interpretation. Changes in fuel costs are largely dictated by the unit cost of fuel and may not be directly linked to changes in fuel use or levels of warmth. In addition, changes in housing costs to recipients, including rent and fuel, may be mediated by welfare provision and changes in the individual's eligibility for welfare benefits such as housing benefit. As such, neither of these changes can necessarily be taken to indicate an improvement, or not, for the recipient of the intervention.

The three studies which presented a ‘balance-sheet’ approach reported a positive net economic benefit following the intervention, based on the outcomes valued in monetary terms.42–44 50 One cost-effectiveness study45 reported that the intervention was more costly and less effective, in terms of SF-36 score, than the status quo,45 indicating that the intervention was not cost-effective. This may reflect the fact that mental and physical health (measured by SF-36) deteriorated following housing improvement; or, and perhaps more likely, that the disruption during housing upgrading led to deterioration in health outcomes and the relatively short period of follow-up (maximum of two years) failed to capture the longer term impacts of the intervention.

The absence of long term health impacts limits the potential for economic analysis. The longest follow-up in this group of studies was 3.5 years after the intervention (range 1 month to 3.5 years). Expectations that health impacts will be observed in this short timescale may be naive and it may be more realistic to hypothesise that the potential for health benefits could be many years after the intervention, perhaps only in the next generation of residents. A full economic evaluation should ideally consider the impacts over the lifetime of the intervention. However, attributing longer term impacts to a historical intervention, even in large scale datasets with minimal attrition, introduces an additional level of confounding. Uncertainty due to immeasurable confounding is an important issue even for short term studies. As such, data on longer term health impacts may be useful but requires careful interpretation.55

The near absence of economic evaluation of housing improvements cannot solely be explained by difficulties in collecting suitable data. Over 40% of the studies which presented cost data alone, had sufficient data to conduct an economic evaluation, but had not done so. This comparative data on costs and health outcome for the intervention and status quo could have been presented to provide a measure of value for money of the intervention, for example in terms of the additional cost per unit of effectiveness achieved. The specific form of the evaluation would depend on the measure of health outcome collected with data such as SF-36, EQ-5D, etc used to determine QALYs for a cost-utility analysis, monetary values of health outcome used within a cost-benefit analysis and other measures of health/clinical outcome used within a cost-effectiveness analysis (see table 1). Where economic analysis was conducted, most (n=3/4) of the studies claimed to have undertaken a cost-benefit analysis but had in fact presented a ‘balance sheet’ approach. This hybrid approach can be helpful for policy-makers by identifying the costs and outcomes associated with a policy/intervention and who bears/receives these impacts.7 51

In each of these three studies, the authors calculated the net economic benefits associated with the interventions of interest, as required to establish value for money in a full cost-benefit analysis. However, none of these studies included changes to health outcomes within the benefits assessment in their calculation, despite all three collecting data on health outcomes. As such, none of these studies provides a full monetary assessment of the benefits associated with the interventions of interest. Instead the economic benefits calculation was restricted to a monetary valuation for increased school and/or work attendance and reductions in CO2 emissions. All three included changes in health service utilisation within the calculation, although these cost changes were frequently misreported as benefits.

These results suggest three important factors. First, the importance of and need for collecting data over a reasonable period of follow-up to allow detection of long term health improvements. Second, the importance of employing wider perspectives through inclusion of costs and savings in other sectors (eg, education or the environment) to give greater potential to show cost offsets and/or cost-effectiveness of housing interventions. Third, and perhaps most importantly, a lack of familiarity with the techniques of economic evaluation such that studies with relevant data often fail to make best use of it, while other studies fail to collect relevant data or misrepresent analyses that are undertaken.

Conclusions and lessons for the future

Rigorous economic evaluation of public health interventions, including those in housing, is seen as a priority. Future studies planning an economic evaluation need to make best use of all available data as well as ensuring that all relevant data are collected. To facilitate this, economic evaluations should be planned alongside the intervention with health economists included from the outset. When undertaken appropriately, economic evaluation provides the potential to make a significant contribution to healthy housing policy.

Economic evaluation assessing the costs and health benefits of healthy public policy interventions has been advocated but has rarely been undertaken.

Housing investment represents a substantial policy investment area which is considered to have potential to contribute to a wider public health strategy to improve population health and reduce health inequalities.

Some studies assessing the health impact of housing improvement have reported some data on costs and economic analysis.

Examination of the nature of cost data and economic analysis reported in these studies may help to explain why economic evaluation has rarely been conducted and provide tangible examples of the potential to develop economic evaluation of healthy public policy investment.

Despite availability of cost data, opportunities to conduct economic evaluations of housing interventions have often been missed.

A small number of studies have reported ‘economic evaluation’ but the term has often been misappropriated.

Future studies planning an economic evaluation should ensure that it is planned alongside the intervention to ensure that all relevant data are collected and available data are fully utilised.

Studies should include a health economist from the outset.

Increasingly policy-makers, seeking to spend money from limited budgets, want evidence of value for money associated with changes in public policy.

Appropriately conducted economic evaluations have the potential to identify public policies which represent good value for money.

Existing economic analyses of the health impacts of housing improvement have often fallen short of undertaking economic evaluations.

Improved planning and better use of available data should ensure that economic evaluations are undertaken which can provide policy-makers with the evidence they require.