Use of relative and absolute effect measures in reporting health inequalities: structured review.

OBJECTIVE: To examine the frequency of reporting of absolute and relative effect measures in health inequalities research.
DESIGN: Structured review of selected general medical and public health journals. DATA SOURCES: 344 articles published during 2009 in American Journal of Epidemiology, American Journal of Public Health, BMJ, Epidemiology, International Journal of Epidemiology, JAMA, Journal of Epidemiology and Community Health, The Lancet, The New England Journal of Medicine, and Social Science and Medicine. MAIN OUTCOME MEASURES: Frequency and proportion of studies reporting absolute effect measures, relative effect measures, or both in abstract and full text; availability of absolute risks in studies reporting only relative effect measures.
RESULTS: 40% (138/344) of articles reported a measure of effect in the abstract; among these, 88% (122/138) reported only a relative measure, 9% (13/138) reported only an absolute measure, and 2% (3/138) reported both. 75% (258/344) of all articles reported only relative measures in the full text; among these, 46% (119/258) contained no information on absolute baseline risks that would facilitate calculation of absolute effect measures. 18% (61/344) of all articles reported only absolute measures in the full text, and 7% (25/344) reported both absolute and relative measures. These results were consistent across journals, exposures, and outcomes.
CONCLUSIONS: Health inequalities are most commonly reported using only relative measures of effect, which may influence readers' judgments of the magnitude, direction, significance, and implications of reported health inequalities.

Introduction

Presenting evidence for the effect of exposures or treatments on health outcomes in exclusively absolute or relative terms can have a dramatic effect on its interpretation and may lead to confusion among patients and physicians about the efficacy and risks of treatments.1 2 3 4 This has led to considerable debate about the ethics of clinical trial design,5 reporting of randomised controlled trial results,6 7 use of evidence based medicine,8 9 10 and communication of absolute risk reductions to patients.11 As a result, two initiatives aimed at improving the reporting of evidence in biomedical literature—the consolidated standards of reporting trials (CONSORT)12 and strengthening the reporting of observational studies in epidemiology (STROBE)13—now recommend reporting both absolute and relative measures of effect whenever possible.

The distinction between relative and absolute effects is also relevant in the context of reporting health inequalities.14 15 16 In both aetiological and descriptive studies, the effect of characteristics such as social class, ethnic background, or geographical location on health outcomes may be quantified in relative or absolute terms. As with other exposures, the choice of effect measure in reporting health inequalities influences our understanding of secular trends and which populations may experience a higher disease burden. In some cases, absolute and relative measures may diverge with respect to the magnitude or direction of change of health inequalities, leading to fundamentally different conclusions. Methodological overviews of measuring and monitoring health inequalities,17 18 including the World Health Organization’s Commission on Social Determinants of Health,19 thus recommend reporting both absolute and relative measures, whenever possible, for properly tracking health inequalities.

To date, no systematic assessment has been done of whether researchers actually follow these recommendations when reporting social inequalities in health. We did a structured review of selected medical and public health literature to determine the frequency of reporting absolute and relative measures of effect on health.

Methods

Search strategy

We did a literature search for all studies in calendar year 2009 that reported quantitative evidence on social inequalities in health, published in 10 leading medical, public health, and epidemiology journals: American Journal of Epidemiology, American Journal of Public Health, BMJ, Epidemiology, International Journal of Epidemiology, JAMA, Journal of Epidemiology and Community Health, The Lancet, The New England Journal of Medicine, and Social Science and Medicine. We consulted a professional health sciences librarian to create a search strategy that combined the above journal titles with Medline medical subject headings (MeSH) search terms and keywords for health inequalities, as well as MeSH search terms and keywords for social inequalities (see web appendix for complete search terms). We included only articles reporting original research results with a quantitative measure of health inequality in the full text. We excluded review articles, systematic reviews, editorials, and commentaries unless they contained original empirical results.

Six reviewers independently reviewed titles, abstracts, and full text. At least two reviewers independently reviewed every article at each stage. All discrepancies were resolved by consensus.

Data extraction and analysis

We developed a standardised data extraction form, adapted from Schwartz et al.20 We piloted the form on 10 articles and subsequently revised it. Reviewers extracted the following information from each article: name of first author, article title, journal title, social group category (socioeconomic position, race/ethnicity, sex, geography, other), type of health outcome (health behaviour, health condition, self reported health, other), and outcome category (prevalence, incidence, mortality/survival). Because disagreement between absolute and relative measures may be more evident in analyses comparing inequalities at different points in time or in populations with different baseline risks, we also created a binary indicator identifying studies that provided some comparative analyses of inequality.

Reviewers recorded the type of effect measure (relative, absolute, both, none) reported in the article abstract and in the full text, tables, and figures of the article, including online appendices. If a study reported absolute measures for some outcomes and relative measures for other outcomes, we categorised it as presenting both absolute and relative measures. We considered odds ratios, risk ratios, rate ratios, hazard ratios, attributable fractions, and summary measures of relative health inequality such as the relative index of inequality or relative concentration index to be relative effect measures. We considered risk or rate differences, attributable risks, and summary measures of absolute health inequality such as the slope index of inequality to be absolute effect measures.17 21 If a study reported only relative measures, reviewers recorded whether the article contained sufficient information to enable calculation of a corresponding absolute measure (for example, whether the absolute risks in each population underlying the relative measure were reported).

Four reviewers independently abstracted data from the articles; each article was independently reviewed by at least two reviewers. All discrepancies were resolved by consensus. Frequencies and summary statistics were calculated by using Stata (version 11). Agreement for all items was high (κ statistic of 0.85 or higher for all items).

Results

The original search returned 1646 articles. We selected articles for relevance in three phases: title, abstract, and full text review. We excluded articles if they did not contain original empirical results or if they did not report at least one quantitative measure of a social inequality in health in the full text. Figure 1 shows the number of articles at each stage of review. Application of our exclusion criteria resulted in a total of 344 articles for inclusion. Table 1 shows the number of articles from each journal included in our sample. The articles in our sample most frequently reported inequalities according to socioeconomic position (53%, 181/344) and race/ethnic background (19%, 66/344), and most articles focused on specific diseases or health conditions as the outcome (68%, 233/344) (results not shown).

Fig 1 Flow diagram of structured review

Table 1

 Descriptive statistics: frequency of articles included for analysis by journal

Journal	No (%)
American Journal of Epidemiology	24 (7)
American Journal of Public Health	71 (21)
BMJ	17 (5)
Epidemiology	1 (0.3)
International Journal of Epidemiology	24 (7)
JAMA	3 (0.9)
Journal of Epidemiology and Community Health	48 (14)
The Lancet	7 (2)
The New England Journal of Medicine	0
Social Science and Medicine	149 (43)
Total	344

Table 2 shows results for type of measure reported in the abstract and in the full text. Overall, 40% (138/344) of articles reported a measure of effect in the abstract. Among these, 88% (122/138) reported only relative measures, 9% (13/138) reported only absolute measures, and 2% (3/138) reported both absolute and relative measures.

Table 2

 Frequency of absolute and relative effect measures reported in abstract and in full text (n=344)

	No	Percentage (95% CI)
Abstract
No measure reported	206	60 (55 to 65)
Only relative measure	122	35 (30 to 41)
Only absolute measure	13	3.8 (1.8 to 5.8)
Both relative and absolute measures	3	0.9 (0.0 to 1.9)
Full text
Only relative measure	258	75 (70 to 80)
Absolute risks not reported	119	46 (40 to 52)
Absolute risks reported	139	54 (48 to 60)
Only absolute measure	61	18 (14 to 22)
Both relative and absolute measures	25	7.3 (4.5 to 10)

In the full text of the articles, 75% (258/344) reported only relative effect measures, 18% (61/344) reported only absolute effect measures, and 7% (25/344) reported both absolute and relative effect measures. Among articles that reported only relative effect measures, 46% (119/258) contained no information on absolute baseline risks that would facilitate calculation of even crude absolute effect measures.

The pattern of reporting of absolute and relative measures of inequality was similar in the subset of studies (n=96) engaged in comparative analysis of inequalities over time or across populations. Among these studies, 75% (72/96) reported only relative measures, 16% (15/96) reported only absolute measures, and 9% (9/96) reported both absolute and relative measures (results not shown).

The proportion of papers reporting only relative effect measures in the full text of articles ranged from 63% to 88% across journals (table 3), with the exception of Epidemiology, which returned a single study. Among studies reporting only relative measures, the proportion reporting absolute baseline risks ranged from 41% to 80% across journals, with the exception of JAMA, which returned two studies (results not shown). Across all categories of exposure and outcome, both absolute and relative measures were reported no more than 10% of the time, and reporting of relative measures alone ranged from 70% to 100% (results not shown).

Table 3

 Frequency of absolute and relative effect measures reported in full text, by journal

Journal	Type of effect measure reported: frequency (%)
	Only relative	Only absolute	Relative and absolute	Total
American Journal of Epidemiology	16 (67)	7 (29)	1 (4)	24
American Journal of Public Health	59 (83)	10 (14)	2 (3)	71
BMJ	15 (88)	1 (6)	1 (6)	17
Epidemiology	0	1 (100)	0	1
International Journal of Epidemiology	15 (63)	6 (25)	3 (13)	24
JAMA	2 (67)	0	1 (33)	3
Journal of Epidemiology and Community Health	38 (79)	7 (15)	3 (6)	48
The Lancet	5 (71)	1 (14)	1 (14)	7
The New England Journal of Medicine	0	0	0	0
Social Science and Medicine	108 (72)	28 (19)	13 (9)	149
Total	258 (75)	61 (18)	25 (7)	344

Discussion

Whether relative or absolute measures of effect should serve as the basis for evaluating and designing health interventions is debated,22 23 24 but reporting guidelines generally recommend reporting both whenever possible. Despite these recommendations, our analysis of 344 empirical articles published in 10 leading journals in 2009 found that a substantial majority reported only relative effect measures (fig 2). We found that nearly 90% of studies with quantitative estimates in the abstract presented only relative measures, and 75% of all articles reported only relative measures in the full text. Only 7% of all articles reported both absolute and relative measures in the full text. The higher frequency of relative measures was consistent across reported social group categories and health outcomes, as well as across the eight journals with more than one article under review. To our knowledge, this is the first structured review of the use of absolute and relative effect measures in reporting health inequalities.

Fig 2 Reporting of relative and absolute effect measures in abstract and full text

Possible reasons for findings

Several possible reasons exist for the higher frequency of reporting only relative measures of effect in studies of health inequalities. Researchers may report relative measures out of convention, as odds ratios and risk ratios in particular are commonly used in epidemiology and public health research. This may stem from historical arguments that relative measures of effect are more desirable for aetiological investigations,25 although the logic behind such arguments has been shown to be flawed.26 The higher frequency of reporting relative effect measures is not specific to health inequalities research. A previous study of general medical journals found that absolute risks were frequently unreported or difficult to access in randomised trials and cohort studies.20

Alternatively, researchers may decide to report only relative measures because these reflect some underlying methodological preference. As they are dimensionless ratios, relative measures may be argued to be more useful than absolute measures for comparing rates across time, space, or disease categories. Low and Low have also argued that relative measures are better for assessing progress in reducing inequalities because, in the context of overall health improvement, narrowing relative measures necessarily imply narrowing absolute measures when health is improving relatively faster among the people who are worse off.23 Finally, researchers may report relative measures alone because they suggest a stronger effect than the corresponding absolute measures.

Implications of findings

Whatever its cause, the high frequency of reporting of only relative effect measures is important for several reasons. Firstly, the choice of absolute or relative measure has been shown to influence conclusions about whether inequalities are increasing or decreasing over time27; which countries, social group categories, or health conditions show the largest inequalities28 29; and whether interventions widen or narrow inequalities.30 For example, one widely cited 1997 study asserted that, despite strong commitments to egalitarian welfare policies, Scandinavian countries had larger socioeconomic inequalities in health than did other Western European nations.31 Scandinavian researchers replied that this was true only if inequality was measured exclusively in relative terms.32 Similarly, several studies have looked at socioeconomic or racial/ethnic inequalities in mortality before and after the introduction of highly active antiretroviral therapy (HAART).33 34 Although the therapy had dramatic effects on reducing mortality from HIV/AIDS in all groups, the relative decline was smaller among some less advantaged groups because of high pre-HAART mortality levels. Studies that looked only at relative differences in mortality found that the introduction of HAART widened inequalities in mortality, but those focusing on absolute mortality differences concluded the opposite.24 35 36

Secondly, experimental investigations have shown that framing health information exclusively in absolute or relative terms can influence decision making. For example, exclusive use of relative risk measures can lead physicians and patients to overestimate the efficacy of a particular treatment.1 2 3 4 Although no direct experimental evidence exists for policymakers’ judgments of health inequalities, one could hypothesise that reliance on measures of relative inequality in the absence of any absolute risk context could lead to overestimation of the magnitude of health inequalities for diseases with low background risks.16 In the same way that lack of access to absolute risks in trials and observational studies can lead to exaggerated perceptions of effects of treatment,20 the low frequency of reporting both absolute and relative effect measures may make it difficult for readers to gain a comprehensive understanding of social inequalities in health. Finally, although both absolute and relative effect measures are technically accurate, exclusive use of one or the other may introduce implicit normative judgments into the assessment of health inequalities. Relative measures express inequality as a dimensionless ratio and thus do not capture information on overall population health or base rates of health. Whether by design or by accident, the high frequency of reporting only relative effect measures may endorse a strict egalitarian position that relative equality is paramount, independent of other considerations.37 38

Limitations of study

Our study has limitations. Firstly, given the volume of original research on health inequalities,39 to obtain a manageable set of results we limited our sample to 10 journals in one calendar year. We selected journals on the basis of reputation, quality, and impact, and believe that they are likely to represent high quality reporting on health inequalities. Nevertheless, adding more journals to our sample may have yielded a different result. Secondly, given more recent attention to absolute inequalities, replicating this study with articles from a subsequent year might produce a different result. Thirdly, we relied on searches of electronic databases of the journals in our sample. The US National Library of Medicine introduced a medical subject heading for “health status disparities” only in 2008, and the ability of this term to capture all articles that would be relevant to our study is unknown. Reviewing the complete editions of these journals may have returned a different set of articles and thus a different result. However, given the high frequency of articles reporting only relative effect measures, and the consistency across journals, outcomes, and exposures, we think it unlikely that an expanded set of articles or journals would produce substantively different results.

Finally, we analysed articles as a whole rather than each measure separately. Thus, we recorded an article that reported an absolute measure for one analysis and a relative measure for another analysis as reporting both absolute and relative measures. Similarly, if an article reporting only relative measures reported corresponding absolute risks for some but not all outcomes, we recorded it as providing absolute risks. If we had examined each measure separately, we may have identified higher proportions reporting only absolute or only relative measures and different proportions providing absolute risks. We do not regard this as a major drawback, as it means that our primary finding of interest—that few articles reported both measures—is probably a conservative estimate.

Conclusion

In our sample of 10 leading medical and public health journals, three quarters of all articles reported only relative measures of effect of social inequalities on health, and roughly half of these provided no information on absolute baseline risks. Despite widespread agreement that it is a best practice for reporting, only 7% of all articles reported both absolute and relative effect measures in the full text. Exclusive reporting of either absolute or relative measures may influence conclusions about whether inequalities are increasing or decreasing over time, the effect of interventions on inequalities, and which countries, social groups, or health conditions show the largest inequalities. In addition, the high prevalence of reporting only relative effect measures could lead to overestimation of the magnitude of health inequalities for diseases with low background risks. Following existing recommendations by reporting both absolute and relative measures will increase transparency, reduce systematic reporting biases, and improve the evidence base for policies aimed at reducing health inequalities.

Choice of absolute or relative measures influences conclusions about the magnitude, direction, significance, and implications of reported health inequalities

Exclusive use of relative or absolute effect measures influences patients’ and health professionals’ interpretation of health information

Reporting guidelines recommend using both absolute and relative effect measures whenever possible

Less than 10% of articles reported both absolute and relative measures of effect, and three quarters reported only relative measures of effect of social inequalities in health

The high prevalence of reporting only relative effect measures may influence conclusions about the magnitude, direction, significance, and implications of reported health inequalities

Following existing recommendations by reporting both absolute and relative measures will increase transparency, reduce systematic reporting biases, and improve the evidence base for policies aimed at reducing health inequalities