Ovarian and cervical cancer awareness: development of two validated measurement tools.

BACKGROUND: The aim of the study was to develop and validate measures of awareness of symptoms and risk factors for ovarian and cervical cancer (Ovarian and Cervical Cancer Awareness Measures).
METHODS: Potentially relevant items were extracted from the literature and generated by experts. Four validation studies were carried out to establish reliability and validity. Women aged 21-67 years (n=146) and ovarian and cervical cancer experts (n=32) were included in the studies. Internal reliability was assessed psychometrically. Test-retest reliability was assessed over a 1-week interval. To establish construct validity, Cancer Awareness Measure (CAM) scores of cancer experts were compared with equally well-educated comparison groups. Sensitivity to change was tested by randomly assigning participants to read either a leaflet giving information about ovarian/cervical cancer or a leaflet with control information, and then completing the ovarian/cervical CAM.
RESULTS: Internal reliability (Cronbach's α=0.88 for the ovarian CAM and α=0.84 for the cervical CAM) and test-retest reliability (r=0.84 and r=0.77 for the ovarian and cervical CAMs, respectively) were both high. Validity was demonstrated with cancer experts achieving higher scores than controls [ovarian CAM: t(36)= -5.6, p<0.001; cervical CAM: t(38)= -3.7, p=0.001], and volunteers who were randomised to read a cancer leaflet scored higher than those who received a control leaflet [ovarian CAM: t(49)=7.5, p<0.001; cervical CAM: t(48)= -5.5, p<0.001].
CONCLUSIONS: This study demonstrates the psychometric properties of the ovarian and cervical CAMs and supports their utility in assessing ovarian and cervical cancer awareness in the general population.

RCT Entities:   Population Interventions Outcomes

Introduction

The UK Government's support for raising public awareness of the early warning signs and risk factors of cancer is set out in the 2007 National Health Service (NHS) Cancer Reform Strategy (see http://tinyurl.com/cancerreformstratergy) and has led to the creation of the National Awareness and Early Diagnosis Initiative (NAEDI). NAEDI aims to promote early diagnosis of cancer with a view to improving the UK's relatively poor survival rates.1 It comprises four work-streams, one of which (research, evaluation and monitoring) has been instrumental in the development of a measure of cancer awareness.2 The Cancer Research UK Cancer Awareness Measure (CAM) was developed to provide a standardised and valid measure of public awareness of cancer, either in whole populations or specific subgroups. It is hypothesised that low levels of cancer awareness in the UK population may be partly responsible for delayed presentation with symptoms, leading to later-stage diagnoses. The CAM was developed to enable researchers and campaigning groups to systematically assess this possibility and to measure the impact of their awareness-raising activities.

This study demonstrates the psychometric properties of the ovarian and cervical Cancer Awareness Measures (CAMs).

The ovarian and cervical CAMs demonstrated good reliability and validity in a series of development studies.

The ovarian and cervical CAMs can be used to assess population levels of awareness, monitor changes in awareness over time and evaluate awareness-raising interventions.

In addition to the generic CAM, there is a need for site-specific versions, particularly for cancers where poor knowledge of risk factors and early warning signs may be associated with delayed presentation and thus have an impact on survival. The aim of the present research was to develop and validate two versions of the CAM that were specific to ovarian and cervical cancer.

Ovarian cancer is the fifth most common cancer in women in the UK with around 6800 new cases each year. Survival rates have increased since the 1970s, with the latest figures showing 5-year survival at 40%.3 Thanks to a successful screening programme, cervical cancer is less common. Around 2800 new cases are diagnosed in the UK each year and 940 women die of the disease, but as with ovarian cancer, survival rates are considerably poorer than in other European countries.4 Thomson and Forman's review of EUROCARE (European Cancer Registry-based study on survival and care of cancer patients) data5 concluded that the survival gap for both ovarian and cervical cancers could be reduced by improvements in early diagnosis.

Survival rates for both cancers vary by stage at diagnosis with better prognosis among women diagnosed with early-stage disease.6–8 Most cervical cancer deaths (80%) occur in women aged over 45 years and the majority of these deaths are among women who have not attended screening. Attending for screening, and (for young girls) receiving the human papillomavirus (HPV) vaccine, are the most effective ways to prevent cervical cancer. However, if cancer does develop, early detection promotes better survival. Several early warning signs for cervical cancer have been identified, including bleeding between menstrual periods, after sex, or after the menopause, and pain during sex.9 Knowledge of these warning signs could improve women's ability to identify the significance of symptoms and may encourage prompt help-seeking.

There is currently no available screening programme for ovarian cancer and only a fifth of ovarian cancers are diagnosed at an early stage in the UK.4 Contrary to previous thinking, recent evidence indicates that there are detectable warning signs and symptoms, which are included in the ovarian cancer ‘Key Messages’ published by the Department of Health in 2009.10 A similar consensus statement on ovarian cancer symptoms was produced in the USA in 2007.11 Public awareness of these warning signs, alongside confidence to present at primary care, could reduce patient-attributable delay and ensure earlier stage at diagnosis.

Smoking and being overweight are potentially modifiable risk factors for cervical and ovarian cancers, respectively. Raising awareness of these risk factors would be a step towards promoting behaviour change and risk reduction.12 Other risk factors are not amenable to change (e.g. family history of breast or ovarian cancer as a risk factor for ovarian cancer, or having a weakened immune system for cervical cancer), but public awareness could facilitate appropriate personal risk perceptions and prompt help-seeking.

The aim of this study was to develop and validate comprehensive measures of ovarian and cervical cancer awareness that would be suitable for identifying gaps in public awareness, monitoring the impact of awareness-raising initiatives, and identifying population subgroups that might benefit from targeted information about these cancers.

Methods

Generation of items

The ovarian CAM and the cervical CAM follow the format of the generic CAM.2 Awareness of warning signs and risk factors is assessed first with an open-ended question and then with a prompted checklist. The list includes all the established warning signs and risk factors, without any distracter items. We identified risk factors and warning signs for ovarian and cervical cancer using the scientific literature as well as cancer information materials and websites. In addition, the Department of Health's ‘Key Messages’ on ovarian and cervical cancer were used to identify the accepted warning signs.9 10

Anticipated time to seeking help for a possible warning sign is assessed on a scale from ‘1–3 days’ to ‘never’. Perceived lifetime risk of cancer is assessed in the generic CAM by asking how many people out of 100 will develop cancer at some point in their lives. Because the lifetime risk of cervical cancer is less than 1 per 100 women this item was adapted for the cervical CAM so that risk was estimated per 1000 women.

Knowledge of the cancer screening programmes is measured in the generic CAM. The cervical CAM assessed awareness of the NHS cervical screening and HPV vaccination programmes and the age of invitation for each. The ovarian CAM used questions from the generic CAM assessing knowledge of all three NHS cancer screening programmes and also asked participants whether there is an ovarian cancer screening programme.

The cervical and ovarian CAMs included an item on awareness of the peak age of incidence of each cancer (now part of the generic CAM) (see http://tinyurl.com/CAMdownloads), and the cervical CAM asked about women's confidence about noticing a cervical cancer symptom.

Expert review

First drafts of the ovarian and cervical CAMs were circulated to a panel of experts (n=5 and 7, respectively), including academic researchers and clinical specialists, to ensure content validity. The panels were asked to comment on the content and comprehensiveness of the questionnaire as well as item terminology and wording. A number of minor changes were made to the draft questionnaires following this review. The majority of comments on the ovarian CAM related to the risk factor section. There was discussion of what the most relevant items were and the state of the current evidence. Specifically, the wording and items that were modified related to going through the menopause, a diagnosis of endometriosis or ovarian cysts, and being childless. There was discussion about the inclusion of ‘using talcum powder in the genital area’ as it was felt that this would not apply to many women. However, because the aim was to maintain a comprehensive list of risk factors, this item was retained because it has been widely publicised as a risk factor and the evidence has not yet ruled it out. However, future users of the ovarian CAM may wish to omit this item if more conclusive evidence becomes available.

In the cervical CAM, ‘infection with herpes’ was removed from the risk factor checklist as experts commented that the evidence for this as a risk factor was inconclusive and ‘taking the contraceptive pill’ was changed to ‘long-term use of the contraceptive pill’. The item ‘not having regular baths or showers’ was removed because experts thought that it was a myth and ‘having a poor diet’ was removed as likely to be a marker of socioeconomic status. The experts also changed the warning sign ‘vaginal discharge’ to ‘persistent vaginal discharge’.

Validating the ovarian and cervical CAMs

The psychometric properties of the measures were assessed in terms of internal reliability, test-retest reliability, item analyses, construct validity and sensitivity to changes in levels of awareness. Parametric statistics (e.g. Pearson's correlation, t-tests) were used to analyse reliability and validity. The open-ended warning sign and risk factor questions were not suitable for use in the reliability and test-retest analyses. However these questions were used in the analyses for construct validity (experts vs control) and sensitivity to change. Data were analysed using SPSS 18.0® (IBM SPSS, Amronk, NY, USA).

The versions of the ovarian and cervical CAMs used in the validation process are summarised in Table 1. They consisted of 11 items on awareness of warning signs for the ovarian CAM and 12 items for the cervical CAM, 11 on anticipated time to seek medical advice in the ovarian CAM and one in the cervical CAM [NB. The ovarian CAM asked about anticipated delay for each symptom as well as a generic item, whereas the cervical CAM used only the generic item “If you had a symptom that you thought might be a sign of cervical cancer how soon would you contact your doctor to make an appointment to discuss it?”], 13 items on risk factors for ovarian cancer and 12 items on risk factors for cervical cancer, one item on age of peak ovarian/cervical cancer incidence, eight items on awareness of NHS screening programmes in the ovarian CAM, and four items on the NHS cervical screening and vaccination programmes in the cervical CAM. The cervical CAM also included an item on confidence in detecting a cervical cancer symptom. The module on ‘barriers to seeking medical advice’, already developed for the generic CAM, was included and can be used as an optional scale in the ovarian and cervical CAMs but is not included in the validation analyses (10 items). Both questionnaires can be requested at http://tinyurl.com/CAMdownloads.

Table 1

Summary of items in each section of the ovarian and cervical Cancer Awareness Measures (CAMs)

Awareness section	Ovarian CAM	Cervical CAM
Warning signs	1 open item	1 open item
	10 prompted items	11 prompted items
Risk factors	1 open item	1 open item
	12 prompted items	11 prompted items
Peak age of incidence	1 item	1 item
Awareness of screening/vaccination programmes	8 items (1 included in the scoring)	4 items
Anticipated time to help-seeking	11 items (one for each warning sign and one for a suspected ovarian cancer symptom)	1 item (for a suspected cervical cancer symptom)
Lifetime risk of ovarian/cervical cancer*	1 item	1 item
Confidence in detecting a symptom	–	1 item
Barriers to help-seeking (optional module from generic CAM)	10 items	10 items

This item showed poor reliability and validity so is not included in the final validated measures.

Samples

Sample 1 (test-retest reliability)

Twenty-two women recruited from among relatives of university staff completed the ovarian CAM via a telephone interview. For the cervical CAM, 22 female postgraduate students were recruited from three non-medical MSc courses. They completed a supervised paper and pencil version of the questionnaire and were monitored to ensure they did not refer back to previous answers. Both samples were selected from age-appropriate groups, that is an older age group (45–67 years) for the ovarian CAM and younger age group (21–40 years) for the cervical CAM, to reflect the age-associated risk for each cancer. These samples completed the measure twice (7–10 days apart) to assess test-retest reliability.

Sample 2 (sensitivity to changes in knowledge)

This sample consisted of female postgraduate students and staff from the university finance department who volunteered in response to publicity about the study. Fifty-two participants completed either face-to-face interviews or supervised paper and pencil versions of the ovarian CAM and 50 participants completed a supervised computer-based version of the cervical CAM. Participants from each group were randomised to read one of two leaflets before completing the CAM; either an educational leaflet on the cancer in question (‘Ovarian/Cervical Cancer: The Facts’) or a control leaflet (‘Recycle to Save the Environment’). There were 27 and 25 participants in the ovarian and cervical intervention groups, respectively, and 25 in each control group. These samples were recruited to test the questionnaire's sensitivity to changes in awareness.

Sample 3 (ovarian/cervical cancer experts)

This sample comprised 13 experts on ovarian cancer and 19 experts on cervical cancer recruited using a ‘snowballing’ method through cancer charities, known gynaecologists and those who had been involved in the development of the measures. Participants took part in telephone interviews for the ovarian CAM and an online version of the cervical CAM. The expert groups were compared with a control group who had equivalent educational qualifications from Sample 2 using the ‘known-groups’ method to establish construct validity. Differences in the scores of two groups known to differ in levels of cancer knowledge support the validity of the questionnaire.13

Item scoring

Warning signs and risk factors

The open questions can be explored in a number of ways depending on the interest of the researcher (e.g. correct vs incorrect responses). In these analyses, one point was given to each warning sign or risk factor that corresponded to an item in the prompted list. This method scores ‘correct’ items. Scores were added to give a total score with the range 0–10 (ovarian) and 0–11 (cervical) for warning signs, and 0–12 (ovarian) and 0–11 (cervical) for risk factors. Higher scores indicate greater knowledge.

The prompted warning signs items were scored as either 0 (no/don't know) or 1 (yes). Scores from each item were summed to produce a total symptom knowledge scale (range: 0–10 for ovarian and 0–11 for cervical), with higher scores indicating greater knowledge. The prompted risk factor items were scored from 1 (strongly disagree) to 5 (strongly agree). A summed scale was produced with higher scores indicating greater knowledge (range: 12–60 for the ovarian CAM and 12–55 for the cervical CAM).

Anticipated time to help-seeking

The items on anticipated time to help-seeking for symptoms were scored individually on an ordinal scale from 1 (1–3 days) to 10 (never). Higher scores indicate greater delay in help-seeking.

Peak age of incidence

This item was scored as correct (score=1) if a participant answered that the age most likely to develop ovarian cancer was either 70 years or 50 years, but incorrect (score=0) for the response ‘30 years old’. While the true answer could be considered to be 70 years, messaging from ovarian cancer charities often focuses on women who are 50 years and older, so both responses were considered valid in this instance. For cervical cancer the item was scored correct (score=1) if the participant answered ‘30–49 years old’.

Awareness of screening/vaccination programmes

In the ovarian CAM, participants were given a score of 1 if they knew that there is not an NHS screening programme for ovarian cancer. In the cervical CAM, participants were given a maximum score of 4 if they were aware of the cervical screening and vaccination programmes, and knew the ages at which these are offered (25 years was the correct response for screening and ages between 12–18 years were counted as correct for vaccination).

Lifetime risk

Answers to this item were recorded verbatim and recoded into correct (score=1) for responses of 1–2 out of 100 for ovarian cancer and 6–10 out of 1000 for cervical cancer, and incorrect (score=0) for other responses.

Total knowledge

For the ovarian CAM, an overall knowledge score was calculated by adding together scores for the prompted warning signs and risk factors, peak age of incidence, lifetime risk and awareness that there is no NHS screening programme for ovarian cancer. [NB. The items on other NHS screening programmes are included to add context to the ovarian screening question but are not included in the scoring of the questionnaire. The questionnaire could be administered without these ‘filler’ items.] This produced a scale where higher scores indicated greater knowledge about ovarian cancer (maximum score=73) [NB. The item on lifetime risk was subsequently dropped from the measure (see below) leaving a maximum score of 72.] For the cervical CAM, the knowledge score was calculated from prompted warning signs and risk factors, age of peak incidence, lifetime risk and awareness of the cervical screening and vaccination programmes and the ages at which they are offered. Higher scores indicated greater knowledge (maximum score=72). [NB. The item on lifetime risk was subsequently dropped from the measure (see below) leaving a maximum score of 71.]

Results

Item analyses

Item discrimination shows whether individual items can discriminate between people with higher or lower knowledge. This test was applied for all items where it was possible to ascertain a correct answer. The items asking about confidence at detecting a cervical cancer symptom and help-seeking were not included as they are not measuring cancer knowledge per se. Items with an item-to-total correlation of >0.2 are considered useful.11 Participants from Samples 1, 2 and 3 were combined in these analyses. Item-to-total correlations for the ovarian CAM were greater than 0.2 for the majority of items. The exception was the warning sign ‘pain in pelvis’ (r=0.19) but this item was retained to ensure content validity because it is part of the ovarian cancer ‘Key Messages’.10 For the cervical CAM, all items had item-total correlations >0.2 except the warning sign item ‘bleeding after the menopause’ (r=0.11). Again, this item was retained because to ensure content validity.

To assess item difficulty we analysed results from Sample 1 and the control group from Sample 2. This analysis allows exploration of item properties and it has been suggested that items answered correctly by 20–80% of respondents should be retained.14 The majority of items in the ovarian CAM fell within this range. A few items received higher rates of correct answers (e.g. identifying pain in the pelvis as a warning sign). The majority of items in the cervical CAM were also answered correctly by between 20% and 80% of respondents. A few items had fewer than 20% answering correctly (‘persistent diarrhoea’ as a warning sign; ‘long term use of the contraceptive pill’, ‘having many children’ and ‘having a sexual partner who is not circumcised’ as risk factors). Additionally, several items were answered correctly by more than 80% of respondents but it was decided to retain all of these items on the basis of content validity.

Reliability and validity

Test-retest reliability

To be sure that questionnaire responses are non-random and stable over time, test-retest reliability was calculated14 using Pearson's correlations between the scores from two time-points. Sample 1 was used to assess test-retest reliability. High correlations (I>0.80) were found for the ovarian CAM as a whole, and for the sections on warning signs, time to seek medical advice, peak age of incidence of ovarian cancer and lifetime risk. Moderate correlations (I>0.55) were found for sections on risk factors and awareness that there is no ovarian screening programme (Table 2).

Table 2

Test-retest reliability of the ovarian and cervical Cancer Awareness Measures (CAMs) (Sample 2, n = 44)

	Test-retest reliability (r)
Awareness section	Ovarian CAM (n=22)	Cervical CAM (n=22)
Warning signs (prompted)	0.83	0.76
Risk factors (prompted)	0.60	0.77
Lifetime risk	0.96	0.58
Peak age of incidence	0.91	0.73
Ovarian screening programme	0.56	–
Anticipated time to help-seeking	0.88	0.92
Confidence noticing a symptom	–	0.55
Cervical screening programme	–	1.00
Age when invited for screening	–	0.80
Vaccination programme	–	0.73
Age invited for vaccination	–	0.81
Total knowledge score	0.84*	0.77†

Total = Prompted warning signs + Prompted risk factors + Peak age of incidence + Ovarian screening programme + Lifetime risk.

Total = Prompted warning signs + Prompted risk factors + Peak age of incidence + Cervical screening programme + Age women invited for screening + Vaccination programme + Age invited for vaccination.

High correlations (r>0.80) were found for the cervical CAM as a whole, as well as sections on help-seeking, awareness of the cervical screening programme, and age invited for vaccination. Satisfactory correlations (r>0.70) were found for sections on warning signs, risk factors, peak age of incidence, age invited for screening and awareness of the vaccination programme. Moderate correlations were found for lifetime risk (r=0.58) and confidence noticing a symptom (r=0.55) (Table 2). Because of its poor reliability in this and other tests, the lifetime risk item was removed from the measure and was not included in the total awareness score for the cervical CAM for this or other analyses.

Sensitivity to change

Data from Sample 2 were used to test whether the measures were sufficiently sensitive to detect the likely modest increase in knowledge achieved by allowing participants to read an information leaflet on ovarian/cervical cancer (compared with a leaflet on recycling). There were no significant differences in age, educational level or ethnic origin between the groups randomised to intervention or control groups for the ovarian CAM. The results showed that after reading the leaflets, the intervention group scored higher than the control group on all the ovarian CAM sections (Table 3) except lifetime risk of cancer. As with the cervical CAM, the lifetime risk item was removed from the measure and was not included in the total awareness score for this or subsequent analyses (Table 3).

Table 3

Differences in Cancer Awareness Measure (CAM) scores between control and intervention participants [t-tests and Chi-square (χ2) tests]

	Ovarian CAM				Cervical CAM
Awareness section	Control (n = 25) Mean (SD)	Intervention (n = 27) Mean (SD)	t (df)	p	Control (n=25) Mean (SD)	Intervention (n=25) Mean (SD)	t (df)	p
Warning signs (open)Max: 10 Ovarian/11 Cervical	0.8 (0.7)	5.2 (4.8)	10.8 (49)	<0.001	0.8 (1.2)	5.1 (2.6)	–7.4 (48)	<0.001
Warning signs (prompted)Max: 10 Ovarian/11 Cervical	4.9 (1.6)	8.4 (2.4)	6.2 (50)	<0.001	5.9 (2.6)	9.3 (2.4)	–4.8 (48)	<0.001
Risk factors (open)Max: 12 Ovarian/11 Cervical	1.6 (1.5)	4.9 (1.6)	7.6 (49)	<0.001	0.8 (0.9)	3.8 (2.1)	–6.4 (48)	<0.001
Risk factors (prompted)Max: 60 Ovarian/55 Cervical	37.6 (3.3)	45.9 (6.5)	5.7 (50)	<0.001	35.7 (5.0)	43.2 (6.2)	–4.7 (48)	<0.001

Total Ovarian CAM knowledge = Prompted warning signs + Prompted risk factors + Peak age of incidence + Ovarian screening programme.

Total Cervical CAM knowledge = Prompted warning signs + Prompted risk factors + Peak age of incidence + Cervical screening programme + Age women invited for screening + Vaccination programme + Age invited for vaccination.

df, degrees of freedom; max, maximum; SD, standard deviation.

Those in the cervical CAM intervention group scored significantly higher than the control group on all sections apart from peak age of incidence, awareness of the cervical screening programme and age invited for vaccination (Table 3) where, although more intervention participants answered the questions correctly, the difference from the control group was non-significant. Demographic data showed that the intervention group had significantly higher educational qualifications than controls, but the significant between-group differences in knowledge persisted after controlling for education (analyses not shown).

Internal reliability

Internal reliability uses Cronbach's α to assess the extent to which all the questionnaire items measure the same underlying construct.14 A score of >0.7 is needed for a scale to be considered internally reliable.15 Samples 1, 2 and 3 were combined for the analysis, and items with a correct response were included (as with the item analyses described earlier). For the ovarian CAM, Cronbach's α of 0.88 was achieved for the knowledge scale (without the lifetime risk item), with the following alphas obtained for each subsection: warning signs=0.77, anticipated time to help-seeking=0.92 and risk factors=0.86.

For the cervical CAM, Cronbach's α for the total knowledge measure (excluding the lifetime risk item) was 0.84, with the following alphas obtained for each subsection: warning signs=0.77 and risk factors=0.77. Other sections in the cervical and ovarian CAMs had too few items to be considered scales. Although responses to open questions can be coded correct or incorrect, this is not a true scale and therefore was not included in this analysis.

Construct validity: cancer experts vs control group

The ‘known-groups’ method was employed to establish construct validity. If the scores of two groups known to differ in levels of cancer awareness are significantly different then the validity of the questionnaire is supported.13 Participants from Sample 2 (control group only) and Sample 3 (ovarian/cervical cancer experts) were used in these analyses. Ovarian and cervical cancer experts scored consistently higher than the group of university staff and students (Table 4). Differences between the groups were statistically significant in all sections of the ovarian CAM (p<0.001). Awareness of the age of peak incidence of cervical cancer, awareness of the cervical screening programme and age of invitation for HPV vaccination did not differ significantly between groups, but a higher percentage of experts answered these questions correctly compared to controls.

Table 4

Differences in Cancer Awareness Measure (CAM) scores between control and expert participants [t-tests and Chi-square (χ2) tests]

	Ovarian CAM				Cervical CAM
Awareness section	Control (n = 25) Mean (SD)	Expert (n = 13) Mean (SD)	t (df)	p	Control (n = 25) Mean (SD)	Expert (n = 19) Mean (SD)	t (df)	p
Warning signs (open)Max: 10 Ovarian/11 Cervical	0.8 (0.7)	6.5 (2.6)	–10.3 (36)	<0.001	0.8 (1.2)	3.6 (1.2)	–7.9 (42)	<0.001
Warning signs (prompted)Max: 10 Ovarian/11 Cervical)	4.9 (1.6)	8.8 (1.9)	–6.9 (36)	<0.001	5.8 (2.6)	8.5 (1.7)	–3.8 (42)	<0.001
Risk factors (open)Max: 12 Ovarian/11 Cervical	1.6 (1.5)	4.5 (1.8)	–5.13 (36)	<0.001	0.8 (0.9)	3.0 (1.6)	–5.7 (42	<0.001
Risk factors (prompted)Max: 60 Ovarian/55 Cervical	37.6 (3.3)	42.1 (4.8)	–3.3 (36)	0.002	35.7 (5.0)	40.0 (6.3)	–2.5 (40)	0.02

Total Ovarian CAMowledge = Prompted warning signs + Prompted risk factors + Peak age of incidence + ovarian screening programme.

Total Ovarian CAMowledge = Prompted warning signs + Prompted risk factors + Peak age of incidence + Cervical screening programme + Age women invited for screening + Vaccination programme + Age invited for vaccination.

df, degrees of freedom; max, maximum; SD, standard deviation.

Discussion

The ovarian and cervical CAMs were developed to provide validated measures of awareness of ovarian and cervical cancer. Reliability of both measures was good, with Cronbach's α of >0.8 and good test-retest reliability for most sections. The only item that performed poorly across the validation studies was the lifetime risk question. Cervical cancer risk was greatly overestimated by both lay and expert groups, with only 21% of experts answering it correctly. It had poor test-retest reliability in the cervical CAM and showed poor sensitivity to change in the ovarian CAM. The item was removed from both measures.

In the ovarian CAM, awareness of screening programmes did not reach the conventional test-retest reliability threshold, but this was not due to major change between the two administrations, but a change from ‘no’ to ‘don’t know' or vice versa. In fact, none of the participants responded ‘yes’ at either time point and therefore this result is relatively unimportant in terms of questionnaire consistency. It does indicate that most women are aware that no ovarian screening programme exists. Responses to the risk factor items also showed only moderate reliability, which may have been a learning effect between the two administrations, because overall risk factor scores were higher in the second round. However, there was still a relatively high correlation, and it is likely that with a longer interval between administrations any learning effect would be diminished if there was not an additional systematic intervention to support it. Future intervention studies using the ovarian and cervical CAMs may wish to explore this further.

Construct validity was established: people with expertise in ovarian/cervical cancer achieved higher scores than an equally-educated control group. This was reinforced by the results from the intervention studies, which showed that both measures were sensitive to increases in awareness following a brief educational intervention.

There were some limitations to this study. The mode of administration varied across stages of piloting, and included face-to-face, telephone, computer-based and paper-and-pencil versions, although all were done under controlled conditions. The exception was the expert panel in the cervical CAM who completed an online version in non-controlled conditions, but it was designed so that participants could not return to previous answers. Experts were also reminded that they should not consult colleagues or search the Internet if they did not know the answer. In addition, the non-expert study participants were better educated than the population as a whole, which limits generalisation to other populations. Future users of the cervical CAM should also be aware of items that are answered correctly by a particularly high or low proportion of respondents. At present, we recommend retaining these items because omitting well-known risk factors would compromise the face and content validity of the measure, and it is important to examine change in the lesser-known items.

Testing of both the ovarian and cervical CAMs was thorough in terms of the types and variety of reliability and validity studies carried out, but further testing in community samples will also be needed to demonstrate utility. Finally, since the development of the ovarian CAM, the National Institute for Health and Clinical Excellence has published draft clinical guidelines for the recognition and initial management of ovarian cancer.16 These state that clinicians should “consider carrying out tests in primary care if a woman reports having abnormal vaginal bleeding” (page 39). Abnormal vaginal bleeding was not included as a warning sign in the ovarian CAM because item choice was driven primarily by the Department of Health's ‘Key Messages’ which, at the time of development, did not include this symptom. However recent evidence has demonstrated that abnormal vaginal bleeding, particularly postmenopausal bleeding, has positive predictive value in relation to ovarian as well as endometrial cancer.17 Therefore future users of the ovarian CAM may wish to consider inclusion of the ‘bleeding after the menopause’ item as used in the cervical CAM. Future users may also wish to add an item measuring confidence in detecting an ovarian cancer symptom which, although not part of the original generic CAM, has been validated in several of the site-specific versions, including the cervical CAM, and has now been included in the generic CAM too (readers are referred to the cervical CAM for the wording of this item).

The ovarian and cervical CAMs are reliable and valid measures of awareness of risk factors and early symptoms of these two cancers. Further tests using community samples are required, but these results indicate that they could be useful tools to provide a comprehensive assessment of ovarian and cervical cancer awareness or to assess the effectiveness of interventions designed to target gaps in awareness of these cancers either in whole populations or specific subgroups.