Explaining variation in cancer survival between 11 jurisdictions in the International Cancer Benchmarking Partnership: a primary care vignette survey.

OBJECTIVES: The International Cancer Benchmarking Partnership (ICBP) is a collaboration between 6 countries and 12 jurisdictions with similar primary care-led health services. This study investigates primary care physician (PCP) behaviour and systems that may contribute to the timeliness of investigating for cancer and subsequently, international survival differences.
DESIGN: A validated survey administered to PCPs via the internet set out in two parts: direct questions on primary care structure and practice relating to cancer diagnosis, and clinical vignettes, assessing management of scenarios relating to the diagnosis of lung, colorectal or ovarian cancer. PARTICIPANTS: 2795 PCPs in 11 jurisdictions: New South Wales and Victoria (Australia), British Columbia, Manitoba, Ontario (Canada), England, Northern Ireland, Wales (UK), Denmark, Norway and Sweden. PRIMARY AND SECONDARY OUTCOME MEASURES: Analysis compared the cumulative proportion of PCPs in each jurisdiction opting to investigate or refer at each phase for each vignette with 1-year survival, and conditional 5-year survival rates for the relevant cancer and jurisdiction. Logistic regression was used to explore whether PCP characteristics or system differences in each jurisdiction affected the readiness to investigate.
RESULTS: 4 of 5 vignettes showed a statistically significant correlation (p<0.05 or better) between readiness to investigate or refer to secondary care at the first phase of each vignette and cancer survival rates for that jurisdiction. No consistent associations were found between readiness to investigate and selected PCP demographics, practice or health system variables.
CONCLUSIONS: We demonstrate a correlation between the readiness of PCPs to investigate symptoms indicative of cancer and cancer survival rates, one of the first possible explanations for the variation in cancer survival between ICBP countries. No specific health system features consistently explained these findings. Some jurisdictions may consider lowering thresholds for PCPs to investigate for cancer-either directly, or by specialist referral, to improve outcomes. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

A novel, large and logistically complicated study using a validated survey.

Data were analysed from 2795 primary care physicians (PCPs) across 11 jurisdictions.

Response rates were suboptimal (ranging from 5.5% in England and British Columbia to 45.6% in Manitoba) and respondents were not totally representative of the PCPs in all jurisdictions.

It is difficult to assess the effect of these weaknesses on the interpretation of results but sensitivity analyses and the literature suggest it would not be large.

Introduction

Significant differences in cancer survival have been demonstrated between countries with similar health systems.1 Poor outcomes may arise from late presentation, diagnostic delays and treatment differences, or combinations of these.1–6 There is some evidence that delay between presentation and diagnosis (the diagnostic interval)7 is associated with poorer outcomes,8–11 but the factors involved are complex and the strength of the relationship is unclear. Detailed knowledge about how the diagnostic interval is managed in health systems may explain these differences in survival.

The International Cancer Benchmarking Partnership (ICBP) is a collaboration across 6 countries (Australia, Canada, Denmark, Norway, Sweden and the UK) and 12 jurisdictions of comparable wealth and universal access to healthcare, established to examine international differences in cancer outcomes and identify possible causes. Cancer survival is higher in Australia, Canada and Sweden, intermediate in Norway, and lower in Denmark and the UK.1 Differences between the countries in the proportion of patients diagnosed with the cancer at an early stage, suggest that differences in the period prior to diagnosis contribute to the international variation in cancer survival, along with other potential factors, such as access to treatment and the quality of treatments.2–6

Public awareness of signs and symptoms, and beliefs about cancer, appear to be quite similar across jurisdictions and are therefore unlikely to account for much of the variation seen between countries. However, differences in perceived barriers to seeing the general practitioner (GP) have been reported.12 Differences in the way cancer symptoms are recognised and managed in primary care may contribute to the observed survival differences. For example, European intercountry differences in clinical diagnostic practice have been reported for gastrointestinal disorders.13 A stronger ‘gatekeeper’ role—whereby primary care physicians (PCPs) manage entry to specialist care and investigations—is also associated with worse cancer survival.14 This ‘gatekeeper issue’ is exemplified by the finding that higher rates of endoscopy referrals within individual UK general practices are associated with a lower mortality from oesophagogastric cancer.15 There are many system factors that will influence a PCPs decision to act, including guidelines, access to investigations, and culture of collaboration between primary and secondary care, and these will all contribute to PCP behaviour.16

The aims of this study were to describe the readiness of PCPs to consider investigation or referral for symptoms possibly indicative of cancer, and to relate this to international differences in primary care structure and practice. Our hypothesis was that there is a positive correlation between the proportion of PCPs who would investigate a specific symptom set for cancer and survival rates (for the given cancer) across jurisdictions. We also investigated whether the readiness of PCPs to investigate these cases could be explained by differences in primary care structure or PCP characteristics.

Methods

We conducted an international vignette survey among a sample of PCPs in participating jurisdictions.

Survey development

We developed an online survey of PCPs exploring differences in their behaviours, attitudes, knowledge and skills relating to cancer diagnosis. Development involved iterative discussion with international partners at every stage of development. The overall validation was initially undertaken in England, with two rounds of validation, using a cognitive interviewing technique with PCPs following completion of the draft survey. Validation of the completed survey was tested in all jurisdictions, particularly to ensure that translation had not altered meaning. There were two questions relating to access to tests and internal consistency was measured by comparison of the answers to these questions. The development and validation has been described in detail elsewhere.17

Structure

The survey was in two parts: first, direct questions consisting of demographic details of the respondents and questions relating to service provision, access to investigations and waiting times following secondary care referrals; and second, vignettes on management choices for a patient presenting with symptoms suggestive of either lung (two vignettes), colorectal (two vignettes) or ovarian cancer (one vignette). Respondents were randomly presented with two vignettes (of different cancer sites), and were asked to complete the survey relating to their own practice rather than perceived best practice. Respondents were aware that the survey was part of a study linked to cancer.

Each vignette had two or three phases: phase 1 represented the first patient presentation; phases 2 and 3 represented subsequent visits, where the patient's symptoms had developed; each phase had a predefined positive predictive value (PPV) for the cancer, based on previous work, to ensure a defined increase in cancer risk at each phase (table 1).18–20 Respondents were not informed of the PPV for each set of symptoms and signs in the survey. However, specifying the PPVs in the analysis at each phase enabled a comparison of the readiness to act within vignettes and between vignettes of the same cancer. The response of primary interest was opting to identify possible cancer either by referral to secondary care or by undertaking a definitive diagnostic investigation in primary care: requesting either of these ended that vignette. The definitive tests were determined by an expert panel and included chest X-ray or lung CT for lung vignettes, colonoscopy or abdominal CT for colorectal vignettes and abdominal CT or abdominal or transvaginal ultrasound for the ovarian vignette.17

Table 1

Summary of the vignettes

Vignette	Cancer	Patient details		Management options		Phase 1		Phase 2		Correlation between survival and referral or test at phase 1 (weighted regression analysis)
				Definitive actions	Non-definitive actions	PPV	Refer/test (%)*	PPV	Refer/test (%)*	1-year survivalp Value	Conditional 5-year survivalp Value
1	Lung	Phase 1	68-year-old woman, exsmoker with persistent cough for 3 weeks but no other symptoms, taking ramipril for hypertension. Ear, throat and chest examinations were normal	Secondary care referralChest X-rayChest CT	▸ Antibiotics▸ Oral steroids▸ Antitussive medicines▸ Stop ramipril▸ Advise visit to pharmacy to try remedies there▸ Tell her that no action is needed at this stage	0.9	40–80	3.9	95–100	0.654	0.647
		Phase 2	The patient returns after another 3 weeks, saying cough has persisted and now there are streaks of blood in the sputum. No weight loss, but a chest examination reveals some crepitations at the left base. Any tests previously undertaken were normal							0.357 (excluding Denmark)	0.911 (excluding Denmark)
		Phase 3	The patient returns with ongoing symptoms and you decide to order a chest X-ray. The report says there is mild cardiomegaly but the lung fields are clear
2	Lung	Phase 1	62-year-old man with COPD, a heavy smoker for over 40 years. He presented with respiratory symptoms	Secondary care referralChest X-rayChest CT	▸ Advise increased use of salbutamol inhaler▸ Antibiotics▸ Oral steroids▸ Antibiotics and add new inhaler steroid or salmeterol▸ Antitussive medicines▸ Advise visit to pharmacy to try remedies there▸ Tell him that no action is needed at this stage	3.6	5–50	>10.0	87–100	<0.001	<0.001
		Phase 2	The patient returns in 3 weeks, reported constant ache in left shoulder. The patient attributes pain to persistent cough; he is also producing grey coloured sputum in larger quantities than usual, but no other chest symptoms. No weight loss. On examination he still has a bilateral upper lobe wheeze and some crepitations at the left base. Examination of his shoulder is normal							<0.001 (excluding Denmark)	<0.001 (excluding Denmark)
3	Colorectal	Phase 1	43-year-old woman with IBS for more than 10 years, but the IBS has got worse recently. She has abdominal pain every day, unchanged bowel habit and no other symptoms. She has no family history of cancer.	Secondary care referralColonoscopyAbdominal CT	▸ Prescribe medication for IBS▸ Give dietary advice▸ Offer psychological therapies (counselling and CBT)▸ Tell her that no action is needed at this stage	0.7	5–45	1.2	48–89	0.014	0.025
		Phase 2	The patient returns. Her recent blood test has returned a haemoglobin level of 10.5 g/dL.							0.003 (excluding Denmark)	0.002 (excluding Denmark)
4†	Colorectal	Phase 1	68-year-old man with no relevant medical history. He has experienced loose stools twice a day, most days for over 4 weeks. He has no other symptoms. Examination included rectal examination which was normal.	Secondary care referralColonoscopyAbdominal CT	▸ Offer medication, eg, loperamide, antispasmodic, analgaesia▸ Advice on diet▸ Tell him that no action is needed at this stage	0.9	20–40	1.9	77–89	0.071	0.093
		Phase 2	Any tests selected are negative. The patient returns 2 weeks later, describing that the diarrhoea remains much the same but he now also has intermittent sharp abdominal pain. Abdominal and PR examinations are normal.							0.004 (excluding Denmark)	0.001 (excluding Denmark)
		Phase 3	All tests are negative. A further 2 weeks later the patient describes two brief episodes of rectal bleeding (bright red) 2 days apart.
5†	Ovarian	Phase 1	53-year-old woman whose last period was 6 months ago. She had experienced abdominal pain for the past 3 weeks. She has had no other symptoms and the same sexual partner for 20 years.	Secondary care referralAbdominal ultrasoundPelvic CT	▸ Prescribe analgesia▸ Prescribe antispasmodic▸ Undertake investigations now▸ Tell her that no action is needed at this stage	0.3	40–75	0.7	68–90	0.007	0.610
		Phase 2	All investigations to date have been normal. The patient presents 1-month later with urinary frequency. She says the abdominal pain is still present but comes less often. Abdominal examination is normal. A urine dipstick for blood, protein, nitrite, white cells and sugar is negative.							0.009 (excluding Denmark)	0.744 (excluding Denmark)
		Phase 3	The patient returns 6 weeks later saying pain is worse, she is passing urine every 3 h, day and night and has noticed that her abdomen seems swollen. After examination of her abdomen, it does look distended but cannot feel any other abnormality.

*Range of respondent who completed the vignette at this stage by referral or undertaking a definitive diagnostic test.

†Stage 3 results not reported as nearly all respondents referred or undertook definitive test by this stage.

Bold typeface indicates significance.

CBT, cognitive behavioural therapy; COPD, chronic obstructive pulmonary disease; IBS, irritable bowel syndrome; PPV, positive predictive value; PR, rectal examination.

The final survey

The survey was completed in 11 jurisdictions: New South Wales and Victoria (Australia), British Columbia, Manitoba, Ontario (Canada), England, Northern Ireland, Wales (UK), Denmark, Norway and Sweden. All surveys were completed online.

Each jurisdiction decided on a method of sampling and approach to potential participants (by post or email), depending on local conditions and the availability of databases with PCP contact details, and participation incentives. While variation in sampling methods and approaches might be expected to introduce sample bias, and to affect response rates, there are no observable trends that would suggest that this is true (see online supplementary table S1).

Participants

Participants were PCPs working predominantly in clinical practice, including locums; retired PCPs and those in training were not eligible. Considering all jurisdictions, the online surveys were completed between May 2012 and July 2013; the start dates and end dates in each jurisdiction varied.

The representativeness of each sample was assessed by comparing gender, age and place of qualification between respondents, and data for all PCPs in each jurisdiction.

All participants gave their consent for their data to be used by completing the online survey. Approvals to conduct this study were sought from ethics committees in each jurisdiction (see online supplementary table S2).

Sample size

Each jurisdiction aimed to recruit at least 200 respondents. This provided a 95% CI ±7% for equally distributed responses (eg, 50% responding ‘yes’), and ±6% CI for less equally distributed responses (eg, 20% responding ‘yes’).

Analysis

Direct questions relating to demographics and primary care structure factors were analysed using descriptive statistics. The questions relating to average waiting times for tests and results were analysed by estimating the mean time from ordering a test to receiving the results by using the midpoint of each waiting time category.

Analysis of the vignettes was undertaken in two stages. Our hypothesis was explored by comparing the cumulative proportion of PCPs opting to investigate or refer at each phase for each vignette using percentage 1-year survival rates in each jurisdiction for the relevant cancer.

Linear correlations were estimated between the proportions opting to investigate at each phase and survival rates.

Weighted regression

When comparing between jurisdictions, weighted linear regression was used to adjust for different sample sizes in each jurisdiction, based on the total number of respondents for each vignette. A regression model was fitted for each phase and each vignette using 1-year survival as the outcome, with the number of PCPs acting at each phase as a single explanatory variable. The weights were the inverse of the variance of the proportion at a given phase for each jurisdiction. No weighted regression models were possible for jurisdictions where all PCPs had opted to investigate at a given phase.

Logistic regression: multilevel model

Weighted regression did not allow us to explore if individual PCP characteristics and access to tests (within each jurisdiction) were associated with opting to investigate. Therefore, in the second stage of the analysis, we fitted separate multilevel logistic regression models using opting to investigate as the outcome and estimated the association with (A) PCP characteristics, (B) access to tests, (C) jurisdiction level survival (1-year and conditional 5-year survival) as reported by Coleman et al,1 and (D) a full model including PCP characteristics and access to tests. This last model only included those variables significantly associated in models (A) and (B) to reduce the chances of identifying spurious associations.

We identified variables that might influence opting to investigate in models (A) and (B), as well as the expected direction of effect (see online supplementary table S3). As with the weighted regression, analyses were performed by vignette as explanatory variables were different for each cancer. These variables are not independent and therefore some basic model selection (based on statistical significant associations p value <0.05) was used to determine the choice of variables included in model (C).

Sensitivity analyses

Previous studies comparing survival rates between England or the UK to other countries suggested that later diagnosis could be a factor contributing to poorer outcomes: thus, opting to investigate or refer later in the process of clinical presentation would contribute to the delay in diagnosis in poorer performing jurisdictions.1 19 Both 1-year and conditional 5-year survival on surviving at least 1 year (conditional 5-year survival) could reflect a longer diagnostic interval due to delays, including in primary care, but neither are a perfect measure of this.21 We chose 1-year survival as the primary outcome measure as we anticipated that it more directly reflects activity in primary care. We then undertook sensitivity analysis using conditional 5-year survival to check that our findings were also confirmed using this outcome.

Denmark has substantially changed its primary care referral procedures since the latest reported comparative survival figures.22 23 This change in procedures was apparent in the results, with Denmark appearing as an outlier in vignettes 3 and 4 related to the diagnosis of colorectal cancer; analyses of these two vignettes were repeated excluding Denmark.

Australia oversampled rural PCPs to ensure the views of this minority were adequately represented. A post hoc sensitivity analysis was undertaken to address this: we formed a data set with a sample of rural PCPs so that this was representative of the rural/urban split in Australia and compared these with the results of the whole data set.

Governance

At all stages, the methodology, sampling and analysis were discussed with four separate working groups: the module lead of each jurisdiction, the Programme Board overseeing the whole ICBP programme, an academic steering group comprising the module chair and three academic PCPs with an interest in cancer (PWR, GR, WH, PV) and an academic reference group comprising primary care academics from within and outside ICBP jurisdictions.

Results

The online survey was completed by 2795 PCPs. Almost all jurisdictions received at least 200 responses; only the Northern Ireland response (the smallest total population) was below this target but with a GP population of 1165 clinicians their crude response rate amounted to 11.2%. Crude response rates (most jurisdictions stopped the survey when the sample size was reached) varied between jurisdictions, from 5.5% in England and British Columbia to 45.6% in Manitoba (see online supplementary appendix table 1).

Characteristics of respondents are summarised in see online supplementary table S4. There were significantly more women PCPs in the samples from Victoria, Ontario and British Columbia and fewer from Norway, when compared with their national data. The samples from England, Wales, Northern Ireland, New South Wales, Victoria, Sweden, Norway and Ontario had fewer doctors who had qualified outside the country, and Manitoba had more (for Canadian provinces and Australian states this meant training outside Canada and Australia, respectively). Respondents from England and Norway were older, whereas respondents from New South Wales and all Canadian provinces were younger, compared with national data. Acknowledging that confidence intervals would be wider than anticipated, Northern Ireland was included for completeness.

Vignette results

In 4 of the 5 vignettes, there was a statistically significant correlation (p<0.05) between opting to investigate for cancer in a jurisdiction at phase 1 and the cancer survival in that jurisdiction (figure 1). Only phase 1 results are presented; nearly all PCPs had taken action to refer or investigate for a potential diagnosis of cancer by the end of phase 2.

Figure 1

Scatterplots of vignettes and multiple regression analysis (PCP, primary care physicians).

Vignette 1 (lung cancer) was the only vignette that did not show any statistically significant correlations. In this vignette, the estimated PPV for lung cancer for the symptoms presented at phase 1 was 0.9% (although this was not explicitly presented in the vignette) and yet the range of respondents opting to investigate at this phase between jurisdictions was 40–80% (table 1). The estimated PPV for lung cancer in phase 1 of vignette 2 was higher at 3.6% and yet the range of respondents opting to investigate at this phase was 4–52%.

The lung and ovarian cancer vignettes were usually completed by the use of a test (chest X-ray for lung cancer, ultrasound for ovarian cancer) rather than by specialist referral. In contrast, more than half of the colorectal vignettes were completed by referral.

Consistent with the correlation results, statistically significant associations were found using weighted regression between the 1-year survival and readiness to investigate at phase 1 for vignettes 2, 3, 4 (after Denmark was excluded) and 5. For the sensitivity analysis based on the conditional 5-year survival, this association was found at phase 1 for vignettes 2, 3 and 4 (after Denmark was excluded; table 1).

Logistic regression analysis

No factors (PCP characteristics, access to tests or length of time from ordering test to receiving results) showed a consistent association with the readiness to investigate. The only two PCP characteristics associated with the outcome were that doctors who trained outside their jurisdiction were more likely to refer or investigate cases earlier in vignettes 3 and 4 (both colorectal) and older doctors were more likely to refer or investigate cases earlier in vignette 4. The former association was investigated further using data from Manitoba (the only jurisdiction to have more doctors trained outside of the jurisdiction); within this data set there was no association found between place of training and readiness to investigate. The latter association was no longer statistically significant when excluding Denmark as part of the sensitivity analysis; while training outside was still significant (data not shown).

Similarly, adjusting for rural/urban split among Australian PCPs did not significantly affect the results.

Direct questions: access to diagnostics and secondary care advice

Similarities and differences in primary care system factors are reported between jurisdictions. Reported direct access to blood tests for cancer diagnosis, plain X-rays and ultrasound was greater than 70% across all jurisdictions (table 2). Access to other tests was variable. Direct access to endoscopy was less common in Canada. The UK and Denmark had comparatively low levels of direct access to CT and MRI scanning.

Table 2

Direct access* to investigations (%)

	Australia		Canada			Denmark	Norway	Sweden	UK
	NSW	Victoria	British Columbia	Manitoba	Ontario				England	Northern Ireland	Wales
Cancer diagnosis blood tests†
Blood tests 95% CI	98.095.2 to 99.3	99.596.6 to 100	81.775.9 to 86.3	69.362.8 to 75.1	80.777.2 to 83.7	89.885.3 to 93.1	70.964.5 to 76.6	75.869.1 to 81.4	94.090.1 to 96.5	89.081.2 to 93.9	75.769.3 to 81.1
Endoscopy
Upper GI endoscopy 95% CI	26.020.8 to 31.9	66.158.9 to 72.8	7.44.5 to 11.8	13.79.6 to 18.9	19.516.4 to 22.9	56.550.1 to 62.6	18.013.2 to 23.5	46.238.9 to 53.2	68.161.9 to 73.8	68.559.1 to 77.2	66.159.3 to 72.2
Flexi sigmoidoscopy 95% CI	17.713.3 to 23.1	34.928.2 to 42.2	13.19.2 to 18.3	13.29.2 to 18.4	17.714.7 to 21.0	54.948.6 to 61.1	16.211.7 to 21.6	34.227.8 to 41.5	40.634.6 to 47.0	14.88.9 to 23.0	32.326.1 to 38.8
Colonoscopy 95% CI	26.421.1 to 32.3	61.954.5 to 68.8	7.94.9 to 12.3	13.29.2 to 18.4	22.419.1 to 26.0	52.546.2 to 58.8	17.112.5 to 22.6	45.237.9 to 52.2	33.127.4 to 39.3	19.813.3 to 29.2	22.016.8 to 28.2
Imaging
X-ray whole body‡ 95% CI	99.296.9 to 99.9	100.097.5 to 100	96.192.4 to 98.1	92.588.1 to 95.5	98.096.4 to 98.9	92.288.0 to 95.0	74.368.1 to 79.8	86.480.6 to 90.7	82.577.1 to 86.9	89.982.3 to 94.6	87.682.3 to 91.5
CT whole body 95% CI	99.697.5 to 100	100.097.5 to 100	92.688.2 to 95.5	86.481.1 to 90.4	95.193 to 96.7	22.017.1 to 27.7	73.567.2 to 79.0	84.378.4 to 89.0	21.516.7 to 27.2	27.519.6 to 37.0	46.339.6 to 53.2
MRI whole body 95% CI	44.938.7 to 51.2	54.046.6 to 61.2	62.055.4 to 68.3	74.668.3 to 80.0	91.689 to 93.6	16.912.6 to 22.2	70.464.0 to 76.2	68.261.1 to 74.5	19.915.3 to 25.5	11.06.1 to 18.8	31.225.2 to 37.9
Ultrasound whole body 95% CI	98.095.2 to 99.3	98.995.8 to 99.8	93.088.7 to 95.8	84.278.7 to 88.6	95.493.4 to 96.9	78.272.4 to 82.9	70.964.5 to 76.6	82.376.1 to 87.2	78.172.4 to 82.9	79.870.8 to 86.7	71.164.5 to 76.9

Exact 95% CI calculated for these figures.

*Direct access: primary care physicians can order the test without specific referral to secondary care.

†Survey did not define which specific tests.

‡Access to all individual body parts.

With the exception of plain X-rays, the total wait from request to receipt of report for imaging or endoscopy was reported to be over 4 weeks in most jurisdictions and over 12 weeks in some. Total wait between a referral for a suspected cancer and a patient's first specialist appointment was between 2 and 3 weeks for all jurisdictions (table 3). Most PCPs reported they could expedite access to tests if cancer was suspected. With the exception of the UK, most PCPs reported ready access to secondary care advice about investigation or referral of suspected cancer (table 4).

Table 3

Average wait times for tests and results (weeks)

	Australia		Canada			Denmark	Norway	Sweden	UK
	NSW	Victoria	British Columbia	Manitoba	Ontario				England	Northern Ireland	Wales
X-ray
Test	0.56	0.57	0.56	0.55	0.54	0.95	1.37	1.43	0.79	0.91	0.82
Result	0.50	0.51	0.71	0.68	0.57	0.79	0.69	0.67	1.03	1.36	1.28
Total wait	1.06	1.08	1.27	1.23	1.11	1.74	2.06	2.10	1.82	2.27	2.10
95% CI	1.00 to 1.12	1.03 to 1.13	1.17 to 1.37	1.15 to 1.32	1.08 to 1.13	1.62 to 1.86	1.91 to 2.21	1.88 to 2.33	1.68 to 1.95	2.01 to 2.52	1.94 to 2.25
CT
Test	0.81	0.82	4.29	4.13	3.01	2.65	3.30	4.07	3.50	5.55	5.38
Result	0.52	0.52	1.02	0.96	0.78	0.98	0.85	1.02	1.39	1.77	1.44
Total wait	1.33	1.34	5.31	5.09	3.79	3.63	4.15	5.09	4.89	7.32	6.82
95% CI	1.24 to 1.42	1.24 to 1.44	4.87 to 5.75	4.69 to 5.49	3.59 to 3.68	3.33 to 3.93	3.88 to 4.43	4.60 to 5.57	4.60 to 5.18	6.57 to 8.07	6.35 to 7.29
MRI
Test	2.39	2.76	12.36	9.10	6.05	5.06	6.13	6.16	4.73	9.04	8.37
Result	0.60	0.56	1.39	1.36	0.88	1.12	1.05	1.24	1.52	2.55	1.56
Total wait	2.99	3.32	13.75	10.46	6.93	6.18	7.18	7.40	6.25	11.60	9.93
95% CI	2.64 to 3.34	2.84 to 3.80	13.12 to 14.38	9.82 to 11.09	6.61 to 7.25	5.66 to 6.69	6.72 to 7.63	6.73 to 8.06	5.90 to 6.60	10.63 to 12.57	8.32 to 30.55
Ultrasound
Test	1.11	1.26	4.30	5.88	1.80	2.71	3.99	3.76	3.73	6.38	6.08
Result	0.52	0.53	0.88	1.04	0.65	0.84	0.78	0.93	1.19	1.62	1.31
Total wait	1.63	1.79	5.18	6.92	2.46	3.55	4.77	4.69	4.93	8.00	7.39
95% CI	1.50 to 1.76	1.56 to 2.02	4.71 to 5.66	6.36 to 7.48	2.32 to 2.59	3.23 to 3.88	4.39 to 5.14	4.25 to 5.13	4.64 to 5.21	7.35 to 8.66	6.90 to 7.88
Upper GI endoscopy
Test	5.56	5.11	9.05	9.35	6.01	2.45	5.95	4.71	3.99	8.37	8.17
Result	1.38	0.83	1.49	1.82	1.36	0.77	0.95	1.46	1.35	1.58	1.67
Total wait	6.94	5.94	10.54	11.17	7.37	3.22	6.90	6.17	5.34	9.95	9.84
95% CI	6.35 to 7.53	5.24 to 6.64	9.85 to 11.72	10.49 to 11.84	1.03 to 7.72	2.97 to 3.47	6.45 to 7.35	5.63 to 6.72	5.02 to 5.66	9.16 to 10.74	9.18 to 10.50
Flexi sigmoidoscopy
Test	5.26	5.06	9.19	9.01	5.79	2.39	6.28	5.55	4.15	8.21	8.20
Result	1.38	0.87	1.55	1.91	1.34	0.76	0.94	1.54	1.46	2.04	1.66
Total wait	6.64	5.93	10.74	10.91	7.14	3.15	7.22	7.09	5.61	10.25	9.86
95% CI	6.06 to 7.22	5.21 to 6.65	10.03 to 11.44	10.20 to 11.63	6.75 to 7.48	2.89 to 3.41	6.76 to 7.65	6.53 to 7.65	5.26 to 5.95	9.35 to 11.15	9.21 to 10.50
Colonoscopy
Test	5.78	5.36	10.48	10.16	6.70	2.51	6.69	6.25	4.15	8.69	9.06
Result	1.38	0.84	9.61	1.91	1.37	0.78	0.98	1.54	1.56	2.00	1.70
Total wait	7.16	6.20	20.09	12.07	8.08	3.29	7.67	7.79	5.71	10.69	10.76
95% CI	6.54 to 7.78	5.49 to 6.91	19.13 to 21.06	11.38 to 12.75	7.72 to 8.43	3.01 to 3.58	7.20 to 8.13	7.16 to 8.41	5.33 to 6.08	9.80 to 11.58	10.08 to 11.44
Average wait time between a referral and first specialist appointment (days)
95% CI	Not available	Not available	15.44	19.21	15.05	4.90	13.70	14.50	10.09	13.53	18.05
	Not available	Not available	14.10 to 16.78	17.56 to 20.87	14.25 to 15.85	4.78 to 5.32	12.72 to 14.68	13.32 to 15.68	9.91 to 10.28	12.62 to 14.44	16.75 to 19.35

Table 4

Access to advice and faster tests (%)

			Australia		Canada			Denmark	Norway	Sweden	UK
			NSW	Victoria	British Columbia	Manitoba	Ontario				England	Northern Ireland	Wales
Percentage agree/strongly agree	Can get specialist advice within 48 h	Investigations	67.4	61.1	67.7	47.8	51.1	80.4	81.8	84.4	31.8	27.6	28.4
		95% CI	61.6 to 73.2	54.1 to 68.1	61.6 to 73.8	41.3 to 54.3	47.1 to 55.1	75.5 to 85.3	76.8 to 86.8	79.3 to 89.5	26.0 to 37.6	19.2 to 36.0	22.4 to 34.4
		Referrals	59.5	64.3	62.5	45.2	44.2	80.7	73.9	79.8	35.8	27.5	25.7
		95% CI	53.5 to 65.5	57.5 to 71.1	56.2 to 68.8	38.7 to 51.7	40.2 to 48.2	75.9 to 85.5	68.2 to 79.6	74.2 to 82.4	29.9 to 41.7	19.1 to 35.9	19.9 to 31.5
	Can arrange faster access to tests, if suspicious	95% CI	87.4	84.7	53.3	64.5	75.5	92.5	82.6	78.8	66.2	70.7	56.0
			83.3 to 91.5	79.6 to 89.8	46.8 to 59.8	58.3 to 70.7	72.0 to 79.0	89.3 to 95.7	77.7 to 87.5	73.1 to 84.5	60.3 to 72.1	62.2 to 79.2	49.4 to 62.6

While there was variation across jurisdictions in terms of access to diagnostics and secondary care advice, none of these factors were associated with PCP readiness to act.

Discussion

Principal findings

Using an online survey in 11 jurisdictions, we have demonstrated a correlation that suggests a relationship between the readiness of PCPs to investigate or refer for suspected cancer and cancer survival in each jurisdiction. This is the first time that readiness to investigate cancer—either directly or by referral to secondary care—has been shown to correlate with cancer survival. Evidence suggests that variations between healthcare systems have an impact on health outcomes.24 There is significant variation between jurisdictions in PCP's access to diagnostic tests. Whether greater access to tests improves outcomes depends on the sensitivity of the test and how the waiting time for test results compares with the waiting time if a referral is made. PCPs may not be aware of the fastest way to diagnose cancer: referral or primary care investigation. Our data indicate significantly long waits in some jurisdictions for the results of tests undertaken in primary care. However, access to tests was not associated with readiness to investigate or refer. Further research is required in this complex area.

Strengths and weaknesses

This was a novel, large, logistically complex survey using a validated tool in 11 jurisdictions with primary care-based health systems. Once respondents engaged in the online survey, the proportion who went on to complete it was high. The vignettes had face validity; they directly reflected clinical practice and were universally applicable.17 Moreover, since the study was completed, the vignettes have been adopted as part of a GP education programme in the UK, in light of their applicability to primary care. There is evidence that using vignettes in a survey correlates well to clinical practice.25 Vignettes have also been used recently in a similar context.26 While surveys using hypothetical vignettes are not ideal, we chose vignette-based surveys as a cost-effective way to maintain consistency across all jurisdictions.

The readiness of PCPs to act consists of personal attributes (eg, knowledge and attitudes about cancer as well as perceptions about the role of PCPs) and system features (eg, guidelines, availability of tests/referral and waiting time for results). A multiple regression analysis did not find a significant association with any of these factors.

This study used an ecological outcome, which makes the risk of an ecological fallacy important. We do not know whether the correlation with readiness and survival is causal or simply an indicator of other factors. Therefore, this study raises the potential of an interaction between the PCPs’ readiness and the system in which they act. These results add to the science pointing towards this important implication.

The poor response rates in some jurisdictions and the lack of representativeness of the respondents in several jurisdictions (based on demographic data) are weaknesses. It is difficult to assess whether and by how much these two factors would affect the interpretation of the results. Two jurisdictions achieved a response rate of over 25%; in other jurisdictions, response rates ranged from 5.5% to 18.7%. A significant obstacle to achieving better response rates in some jurisdictions was limited availability of comprehensive and up-to-date email addresses for PCPs. The response rates achieved are broadly in line with lower response rates for online surveys generally.27 Furthermore, response rates in physician surveys have been declining in recent years.28

Those responding were not wholly representative of their local PCP population, but the differences in gender and country of qualification were bidirectional. Evidence from Australia suggests that while respondents had more positive views about cancer compared to non-responders, the magnitude of this difference is the same irrespective of incentives (conditional or otherwise).29 In addition, respondents were aware the survey was part of a study linked to cancer for ethical reasons; responses might have been different in a blinded study. Taken together, these biases tend to underestimate the possible variation, and our results are thus minimum estimates of the correlation between readiness to investigate and survival. In jurisdictions where respondents were less representative of the local PCP population, respondents were in general more likely to be women, to have qualified in that country and to be younger. Although training outside the country was associated with opting to act earlier in the colorectal vignettes, sensitivity analysis using Manitoba data suggested this is unlikely to have substantially affected overall results. A systematic review of studies of primary care referrals suggested that less than 10% of the observed variation in referral rates could be accounted for by practice and GP characteristics.30 It is therefore unlikely that the minor unrepresentativeness of some samples has materially altered the findings. Other hidden confounders may have influenced the results, but they are unlikely to have been major.

Earlier diagnosis could lead to perceived better outcomes through two mechanisms: earlier stage at diagnosis or lead time bias. Our correlations were significant for both 1-year and conditional 5-year survival, suggesting that lead time bias is less of an issue.

The vignettes all described hypothetical patients with symptoms that have been shown to be significantly associated with the given cancer.

The survey was conducted in 2012/2013 and therefore reflects clinical practice at that time. The latest comparative survival data for the three cancers in participating jurisdictions is from 2007. The jurisdictions had no major changes to cancer diagnostic pathways during that time, with the exception of Denmark, which in 2009, introduced significant reforms to improve diagnostic access for PCPs. Consequently, Denmark was often an outlier in the analyses: sensitivity analyses including and excluding Denmark strengthened our findings.

Comparison with other research

An analysis of the health systems in the ICBP jurisdictions showed few significant differences.15 Poorer outcomes have been correlated with the strength of the ‘gatekeeping’ role of PCPs in different health systems13: our findings broadly support this. Denmark has deliberately reorganised cancer diagnostic services to reduce the gatekeeping role by enabling PCPs to undertake timely investigation of patients with alarming as well as vague symptoms.23 Our results suggest that Danish PCPs now behave more similarly to PCPs in jurisdictions with better cancer outcomes. Gatekeeping may encourage PCPs to overuse other diagnostic strategies, such as the ‘test of time’, which could contribute to longer diagnostic intervals.31 Other possible reasons to account for our results include PCPs’ knowledge and PCPs’ relationships with specialists.

Implications

In this study, PCPs opted to investigate at low levels of risk, possibly reflecting bias, because this was a survey relating to cancer. However, recent work suggests that patients prefer to be investigated when cancer is a possibility even at low-risk levels.25 If risk thresholds at which symptoms are investigated were to be lowered, health economic considerations would need to be taken into account. The use of risk prediction tools32 33 may aid PCPs in this respect. Our findings also suggested that PCPs were not necessarily aware of the PPVs of groups of symptoms. Across all jurisdictions, the speed of referral was inversely related to the PPV for lung cancer in the two lung vignettes. However, these counterintuitive responses are consistent with referral guidelines in some of the jurisdictions. Variation in readiness to investigate or refer to secondary care by PCPs for patients with a differential diagnosis of cancer might explain some of the variation in cancer survival between ICBP jurisdictions. It is unlikely that a single solution to this variation will work across all jurisdictions. However, solutions are likely to include initiatives that empower PCPs towards earlier investigation of cancer and to reduce the barriers that inhibit specialist referral. Such changes are likely to require changes in local policy leading to increased access to investigations and diagnostics, more efficient referral pathways and the redrafting of local referral guidelines, to facilitate referral at risk levels below those currently mandated.

Future research

The study supports the ecological findings that there is a correlation between the healthcare system and the way PCPs perform clinical diagnosis. Therefore, it seems appropriate to perform studies to assess which factors affect the readiness of a PCP to investigate or refer, for example, changed access to investigations, the nature and recommendations of clinical guidelines, access to rapid diagnostics and referral, and the nature of relationships between primary and secondary care. Further studies on using alternate outcome measures such as stage distribution, and cohort studies on survival and mortality, could provide additional insights into these factors.