Does social disadvantage affect the validity of self-report for cervical cancer screening?

OBJECTIVE: The aim was to review the international literature on the validity of self-report of cervical cancer screening, specifically of studies that made direct comparisons among women with and without social disadvantage, based on race/ethnicity, foreign-born status, language ability, income, or education.
METHOD: The databases of Medline, EBM Reviews, and CINAHL from 1990 to 2011 were searched using relevant search terms. Articles eligible for data extraction documented the prevalence of cervical cancer screening based on both self-report and an objective measure for women both with and without at least one measure of social disadvantage. The report-to-record ratio, the ratio of the proportion of study subjects who report at least one screening test within a particular time frame to the proportion of study subjects who have a record of the same test within that time frame, was calculated for each subgroup.
RESULTS: Five studies met the extraction criteria. Subgroups were based on race/ethnicity, education, and income. In all studies, and across all subgroups, report-to-record ratios were greater than one, indicative of pervasive over-reporting.
CONCLUSION: The findings suggest that objective measures should be used by policymakers, researchers, and public-health practitioners in place of self-report to accurately determine cervical cancer screening rates.

Introduction

Detection of invasive cervical cancer by screening confers a better prognosis and improved survival compared with cervical cancer being detected symptomatically.1 This effect is not attributable to lead-time bias, and is largely due to screen-detected cancers being found at earlier stages than symptomatic cancers.1 Due to the clear benefits of cervical cancer screening, and the potential harms when screening is inadequate, policy makers, researchers, and public-health practitioners need valid methods of determining Papanicolaou (Pap) test rates. Underestimation of rates could lead to policies and programs that make inappropriate use of resources, and overestimation could lead to missed opportunities to increase screening.2,3 Self-report of previous tests is a commonly used and relatively easy method to determine prevalence of cancer screening for policymakers and knowledge users, but its validity has been questioned in the literature.4–7

Women with certain sociodemographic characteristics have been found to be under-screened for cervical cancer, including racial/ethnic minorities, immigrants, women of low income, and those with low educational attainment.8–15 The determination of accurate screening rates is of particular importance for women in these groups, to allow for accurate monitoring and evaluation of targeted interventions.16–18 However, the validity of self-report may be of particular concern for these same groups, and the literature is not clear on the subject. For example, the tendency to give socially desirable responses may be higher among those with less education and among non-White survey respondents, but lower among the unscreened.19 As well, previous reviews of the literature have suggested that Hispanic and African-American women disproportionately over-report Pap test screening compared with their White counterparts,3,7 and contradictorily, that minority or low socioeconomic status is not associated with accuracy of Pap test recall.6 The purpose of this study is to review the international literature on the validity of self-report of cervical cancer screening, with a specific focus on studies that make direct comparisons of validity among women with and without social disadvantage.

Methods

The databases of Medline, EBM Reviews, and CINAHL from 1990 to 2011 were searched using the search terms “self report,” “cancer screening,” “early detection of cancer,” “vaginal smears,” “Pap test,” “reproducibility of results,” and “validity”. The search was limited to articles published in English. The reference lists of previous literature reviews3,6,7,20 were also searched to identify studies that may have been missed. This strategy produced 114 unique articles. After review of the titles and abstracts, the authors were left with 24 studies that appeared to validate self-report of cervical cancer screening against an objective measure. The full text of the articles was then reviewed to determine which were eligible for data extraction. Articles eligible for data extraction had to: (1) include the prevalence of cervical cancer screening based on both self-report and an objective measure, ie, medical charts, administrative data, laboratory reports, and/or a cytology registry, for the same women within the same time frame, and (2) had to do so for both women with and without at least one of the chosen measures of social disadvantage. Specifically, social disadvantage was defined based on any of the following: income, education, race/ethnicity, language ability, and foreign-born status. No geographic limitations were set on the literature search.

Five studies met the extraction criteria.21–25 Of note, one study in the literature search26 compared self-report of cervical cancer screening with medical records for women from each of five racial/ethnic groups; however, the study sample only included women who reported a Pap test, meaning the prevalence of positive self-report was artificially set at 100%. Therefore, this study was excluded from this review. The following data were extracted from all studies meeting the selection criteria: setting, number and age range of women in the validation sample, objective measure used, time frame for recall, and the sociodemographic groups that were compared.

The report-to-record ratio, the ratio of the proportion of study subjects who report at least one screening test within a particular time frame to the proportion of study subjects who have a record of the same test within that time frame, is frequently used as a measure of net bias of self-report, with values greater than one indicating over-reporting and values less than one indicating under-reporting.27 Therefore, for each study subgroup, a report-to-record ratio was calculated. SAS version 9.3 (SAS Institute, Cary, NC) was used for all confidence interval calculations.

Results

The five included studies varied widely in the setting, number and age range of included women, and time frame for recall21–25 (Table 1). Four were performed in the US, and one was conducted in Ontario, Canada.24 One study was done at the population level,24 whereas the remainder had sample sizes ranging from 251 to 4784 women. Three studies21,23,25 compared self-report with medical charts and two22,24 with administrative data. Four studies21–23,25 created subgroups based on race/ethnicity, one21 categorized based on educational attainment, and one24 on neighborhood income quintiles (one study21 examined both race/ethnicity and education). No studies examined language ability or foreign-born status.

Table 1

Description of the five studies meeting inclusion criteria

Study	Study setting	Time frame for recall	Age range	Self-report measure	Objective measure	Comparison groups	n
Caplan et al21	Health maintenance organization in Colorado, USA	3 years	40–74	Telephone survey	Medical chart	Minority	40
						White	405
						Hispanic	36
						Non-hispanic	407
						<high school	33
						≥high school	411
Fiscella et al22	Throughout USA	1 year	65+	In-person interview	Administrative data	Minority	634
						White	4150
Hiatt et al23	Health maintenance organization in California, USA	2 years	35–74	Telephone survey	Medical chart	Hispanic	398
						White	288
Wang et al24	Throughout Ontario, Canada	3 years	18+	Telephone survey	Administrative data	Q1 (lowest income)	Not provided; population-level
						Q2
						Q3
						Q4
						Q5
Tumiel-Berhalter et al25	Two family practice clinics in Buffalo, New York, USA	3 years, ever	40+	In-person interview	Medical chart	3 years
						African-American	95
						Puerto Rican	96
						White	60
						Ever
						African-American	112
						Puerto Rican	117
						White	75

In all studies, and across all subgroups, report-to-record ratios were greater than one, indicative of pervasive over-reporting (Figure 1). Over-reporting was especially high in the study with the oldest population, which also had the lowest screening rates.22 Confidence intervals for women with social disadvantage frequently overlapped with confidence intervals for those without, with one exception:22 where minority women had much higher report-to-record ratios. In general, the highest report-to-record ratios were not associated with any particular subgroup, but were associated with low values for the denominator (objective records of Pap tests). The considerable heterogeneity between the five studies precluded meta-analysis.

Figure 1

Report-to-record ratios by study and comparison groups.

Notes:aPercentage of women who reported a Pap test during the outcome time frame; bpercentage of women who had a Pap test recorded during the outcome time frame; csample size not provided; therefore unable to calculate confidence intervals; dunable to calculate confidence intervals due to lack of convergence.

Discussion

In this review of studies that allowed for direct comparison of self-report validity of cervical cancer screening for women with and without social disadvantage, it was found that over-reporting of screening is widespread, regardless of sociodemographic characteristics. The small number of heterogenous studies eligible for review precludes firm conclusions about differential rates of over-reporting by sociodemographic status. Of note, the study that was excluded26 similarly found that only 69.4% of self-reported Pap tests were validated in medical records, with a range of 65.9% for Latina women to 85.1% for White women. These findings suggest that health equity researchers, public-health practitioners, and policy makers must interpret the accuracy of screening rates and screening gaps based on self-report with caution. The finding of widespread over-reporting, particularly when prevalence was low,22 is likely applicable to other kinds of cancer screening and preventive health maneuvers, and this possibility should be explored in similar reviews of the literature.

This review makes several contributions to the literature. To the authors’ knowledge, this review is the first to identify studies that made direct comparisons of cervical cancer screening validity for women with and without social disadvantage. The study reviewed the international literature and investigated measures of social disadvantage beyond race/ ethnicity. It has also highlighted gaps in the literature that need to be addressed. Only a small number of studies were found, most of which had small sample sizes of disadvantaged women. These small sample sizes may in some cases have led to wider and overlapping confidence intervals. There were no studies assessing self-report validity for foreign-born women. Considering the ethnically and socioeconomically diverse nature and the high population of immigrants in many countries, more research studies need to be done to validate self-report for this vulnerable group, especially as known screening inequities may be larger than what would be suggested by self-report.

This review also has limitations. Two different objective measures were used in the included studies, and neither are necessarily perfect gold standards. Medical charts may miss some screening tests, as health care providers may not always record them properly,28 and administrative data also may not capture all screening tests.24,29 As well, socially disadvantaged women may be more likely to visit health centers where physicians are salaried and have less motivation for accurate administrative documentation.3,22,26 However, as these measures are recorded at the time of screening, they are still expected to be more accurate, and more appropriate gold standards, than self-report. As well, the findings held regardless of the type of objective measure used. There were only a small number of studies in the review, with noticeable heterogeneity, precluding a meta-analysis. However, it was possible to calculate report-to-record ratios and confidence intervals for all studies. The paucity of available literature suggests that more studies need to be conducted to address this important research question.

Despite these limitations, the findings suggest that objective measures, such as medical charts, administrative data, laboratory records, or cytology registries, may be better than surveys or questionnaires to more accurately determine both cervical cancer screening rates and the magnitude of screening inequities. Using objective measures also circumvents the issue of response bias. Self-report rates can only be provided for women who agree to report. In situations where there is funding available and support at the policymaker level, organized screening programs linked to complete population-level cytology registries could lead to both more accurate determination of screening rates and gaps, and greater opportunities to increase screening for disadvantaged women. Where such organized programs exist, the infrastructure should be put in place to ensure that screening rates for the general population and for particular subgroups are determined directly from objective measures provided by the program itself. Such programs would also allow for true gold standards for determining screening rates. Of note, neither of the two countries where the included studies took place currently have national organized screening programs.

Where funds for such programs may not be available, and where self-report may be the method chosen due to feasibility or cost, especially when there is limited access to administrative data, ways of improving the accuracy of self-report through improving the wording and structure of surveys needs to be explored.3,24 When possible, researchers should conduct validation studies aimed at determining the requisite mathematical correction factors for various jurisdictions and sociodemographic groups if applicable. With more accurate measures of cervical cancer screening, researchers, primary-care providers, and policymakers will better be able to efficiently and effectively address the screening inequities that exist for disadvantaged women.