Success Rates for the Objectives of US State Cancer Control Plans: A First Look.

BACKGROUND: A well-designed cancer control plan is an important tool for a nation, state, or community to address the burden of cancer. Furthermore, it provides the opportunity to devise and implement measurable objectives. However, there has been little to no assessment of the success rates of such objectives.
METHODS: I compared the success rate of objectives between US states' current plan and most recent past plan to determine the proportion of success in the United States overall. I also tested possible reasons for low success rates.
RESULTS: The mean success rate was 20% for stringent successes (only exact matches between plans) and 28% for loose successes (exact and similar matches between plans). The magnitude of change in percentage between the baseline and target for loose objectives significantly predicted success (P = .0347). Higher change resulted in lower success. However, neither the number of objectives nor the level of overlap significantly predicted success rate. Nor was population size, region, or rural-urban status significantly related to success rate. The most successful states had high proportions of objectives that were measurable and a high number of overlapping objectives.
CONCLUSION: I found that objective success rates were low for cancer control plans. To improve success rates, I suggest that future cancer control plans ensure each objective has a measurable baseline and realistically attainable target.

Introduction

Cancer control plans help address the impact of cancer on populations. In 1998, the US Centers for Disease Control and Prevention (CDC) established the National Comprehensive Cancer Control Program (NCCCP) to assist US states in creating and implementing such plans.[1] Most plans include an overview of state cancer rates; cover relevant topics such as prevention, screening, treatment, and survivorship; and include goals that are measured by objectives. There are currently 66 coalitions with a cancer control plan, which includes all fifty US states, the District of Columbia, eight tribal organizations, and seven US territories/jurisdictions.[2] As of June 2021, most US states have created at least two versions of a cancer control plan, with plans typically spanning 5 years.

Despite the utility of a successfully implanted plan, there appears to be little research on the effectiveness of state-level cancer control plans. Several articles have addressed general topics, such as the implementation of evidence-based practices or the inclusion of specific material such as genomics, capacity/sustainability, or end-of-life content.[3-9] Other articles have focused on general outcomes such as better quantified goals coupled with associated costs of implementation, comparing state-level recommendations to federal recommendations, identifying keys to success,[2] and developing core domains.[10-12]

The only reports I was able to find on explicitly measuring success rates of cancer control objectives are as follows. The first was a broad-level US review that found 52% and 63.8% of cancer control plans had met at least one established annual objective based on an environmental or system change in years 4 and 5 of the plan.[13] The second was a study that found that breast cancer objectives that had a measurable baseline and target in US state plans for incidence, mortality, and mammogram prevalence were at 2%, 21%, and 79%, respectively.[14] The third was a study that found that South Carolina’s cancer control plan had 33% of objectives that were clear and measurable, 38% that were not met, 38% that were partially met, and 24% that were met fully.[15] Apart from those three articles, I was unable to locate any further research on cancer control objective success rates. More importantly, I were unable to find any research that explored the mechanism behind cancer control objective success.

Such a lack of research on successful objective achievement is a glaring omission. The inclusion of specific, measurable, attainable, realistic, and time-bound (SMART) objectives has been emphasized by several resources.[16-18] For its part, the NCCCP’s self-assessment tool for cancer control plans suggests having cancer control objectives: (1) present, (2) clearly labeled, (3) specific, (4) measurable, (5) attainable, (6) result-oriented, and (7) time-phased.[19] Finally, NCCCP’s priorities include (1) emphasizing primary prevention of cancer, (2) promoting early detection and treatment, (3) supporting survivors and caregivers, and (4) promoting cross-cutting priorities.[20] The cross-cutting priority of note is demonstrating outcomes through evaluation. It seems necessary, therefore, to evaluate the success rates of cancer control plan objectives.

Methods

State cancer control plan documents were obtained from the CDC comprehensive cancer control plan publication list (https://ftp.cdc.gov/pub/publications/CANCER/ccc/). If a state did not have at least two plans present on the list, that state’s website was searched for any additional cancer control plan documents. For states with more than two plans, only the current plan and most recent past plan were used. For each state, “Plan Past” is defined as is the most recent past plan, while “Plan Current” is the current plan.

Each cancer control plan document was searched for objectives. The description of the objective was extracted along with the baseline and target measurement. Multifaceted objectives—objectives that contained multiple measurements, like screening rates across three ethnicities—were split into separate objectives, each with a single measurement for the baseline and target. Only “linked” objectives—measurable objectives that matched across Plan Past and Plan Current—were included. A set of linked objectives was labeled “stringent” if it had an identical definition for its measurement criterion. Otherwise, it was labeled “non-stringent” if it had similar but not identical measurement criterion. The proportion of “stringent successes” per state was calculated as the number of stringent successful linked objectives divided by the total number of stringent linked objectives. The proportion of “loose successes” per state was calculated as the number of stringent and non-stringent successful linked objectives divided by the total number of stringent and non-stringent linked objectives. The overall mean proportion of stringent and loose objectives was the average across all states that had attainable data (Table 1; see Supplemental Methods for additional information on classification).

Table 1.

Terminology for Methods.

Name	Description	Example
Plan Past	Most recent past cancer control plan document for a state	ND CCC plan 2011–2016
Plan Current	Current cancer control plan document for a state	ND CCC plan 2018–2022
Linked objective	Objective found in both plans	Decrease current smokers by 2016Decrease current smokers by 2022
Stringent objective	Objective has identical measurement criteria between both plans	Increase age-appropriate Pap test ratesIncrease age-appropriate Pap test rates
Non-stringent objective	Objective has similar measurement criteria between both plans	Increase CRC screening for 50+Increase CRC screening for 50–75
Stringent successes	Number of stringent objectives that were successful	8 stringent = 8
Stringent success rate	Stringent successes divided by all stringent objectives	1 (successes)/20 (total) = .05
Loose successes	Number of stringent and non-stringent objectives that were successful	8 stringent + 2 non-stringent = 10
Loose success rate	Stringent and non-stringent successes divided by all stringent and non-stringent objectives	1 (stringent) + 3 (non-stringent)/20 (total stringent and non-stringent) = .20

I generated three plausible reasons for low objective success rate. The first was a large difference between the baseline measurement and target measurement of objectives. An unrealistically high goal (e.g., hit a target of 80% screening rates from a baseline of 40%) is expected to limit success. The second was an excessive number of objectives. Cancer coalitions have limited resources, so putting insufficient effort into too many avenues is also expected to limit success. The third was low linkage between objectives across time. If an objective in an earlier cancer plan is not included in a later cancer plan, success cannot be measured.

To test the first reason—a large difference between the baseline and target—each linked objective was re-coded as a binary variable called success status (success = 1, non-success = 0). Then, the percentage change between the base measurement and the target measurement was calculated. Measurements that were not percentage-based were excluded from the analysis because they could not be properly scaled. For stringent and loose objectives separately, the success status (1 or 0) was modeled as a function of the percentage change with logistic regression using a binary distribution.

To test the second reason—a high number of objectives —success rates for each state’s objectives were transformed to fit a beta distribution by adding or subtracting a marginal value (.001) to ensure the values did not include exactly 0 or 1. The proportion of stringent successful objectives was then modeled as a function of the number of the measurable objectives for Plan Past and the measurable objectives for Plan Current with a generalized linear model using a beta distribution. The same was done for loose successful objectives. The proportions of successful objectives (stringent and loose) were also modeled as a function of the change in measurable objectives between Plan Current and Past.

To test the third reason—low linkage between objectives across plans—stringent success rates were modeled as a function of the number of stringent matched objectives with a generalized linear model using a beta distribution. The same was done for loose success rates and loose-matched objectives.

Finally, to explore if geographic or demographic data affected success rates, state-level urban population percentages for 2010 were obtained from the Iowa Community Indicators Program (https://www.icip.iastate.edu/tables/population/urban-pct-states). States with urban percentages less > 70% were designated as urban, otherwise they were designated as rural. Next, state population size was taken from the 2010 US Census and log-transformed to approximate a normal distribution. After that, states were also categorized as belonging in one of five regions (Northeast, Midwest, West, Southwest, and Southeast). Finally, stringent and loose success rates by state were each modeled as a function of log population size (numerical) with a generalized linear model using a beta distribution. The same was done for region (categorical), urban percentage (numerical), and rural–urban status (binary).

Results

While all fifty states had at least one cancer control plan, not all states had enough data to be used. Seventeen states out of fifty had no measurable, linked objectives and so could not be used in the analysis. A further two states had no stringent, linked objectives, but at least one loose, linked objective, and so could be used in part of the analysis. Thirty-one states had both stringent and loose objectives and so could be used in the full analysis. Of those states, ten had zero stringent successes and four also had zero loose successes (Table 2).

Table 2.

State Summaries of Cancer Plan Objectives. Past Plan Indicates the Most Recent Past Plan if There Were More than One Past Plan.

State	Past Plan Objectives (Measurable)	Current Plan Objectives (Measurable)	Matched Objectives (Loose)	Proportion Success (Loose)
Alabama	57 (36)	27 (27)	10 (12)	.20 (.25)
Alaska	28 (16)	43 (26)	6 (8)	.00 (.00)
Arizona	35 (3)	25 (19)	0 (0)	N/A (N/A)
Arkansas	38 (26)	67 (51)	9 (13)	.44 (.38)
California	40 (32)	0 (0)	0 (0)	N/A (N/A)
Colorado	96 (74)	0 (0)	0 (0)	N/A (N/A)
Connecticut	70 (27)	13 (0)	0 (0)	N/A (N/A)
Delaware	63 (0)	46 (0)	0 (0)	N/A (N/A)
Florida	71 (10)	64 (39)	4 (5)	.00 (.00)
Georgia	29 (13)	0 (0)	0 (0)	N/A (N/A)
Hawaii	3 (0)	15 (7)	0 (0)	N/A (N/A)
Idaho	35 (33)	35 (31)	19 (24)	.16 (.29)
Illinois	24 (0)	31 (17)	0 (0)	N/A (N/A)
Indiana	55 (38)	30 (24)	4 (6)	.25 (.33)
Iowa	67 (67)	45 (40)	17 (17)	.29 (.29)
Kansas	42 (33)	57 (44)	6 (11)	.33 (.36)
Kentucky	54 (34)	45 (37)	13 (18)	.23 (.28)
Louisiana	114 (61)	17 (12)	0 (0)	N/A (N/A)
Maine	81 (48)	21 (20)	3 (6)	.33 (.33)
Maryland	118 (60)	74 (71)	17 (22)	.06 (.14)
Massachusetts	54 (41)	37 (29)	6 (8)	.17 (.13)
Michigan	144 (68)	42 (40)	1 (4)	.00 (.25)
Minnesota	67 (63)	26 (17)	3 (3)	.00 (.00)
Mississippi	59 (18)	19 (10)	0 (0)	N/A (N/A)
Missouri	32 (19)	54 (48)	5 (7)	.40 (.29)
Montana	36 (29)	0 (0)	0 (0)	N/A (N/A)
Nebraska	37 (26)	31 (26)	0 (3)	N/A (.00)
Nevada	38 (11)	71 (61)	2 (5)	.50 (.20)
New Hampshire	41 (29)	23 (14)	3 (5)	.00 (.00)
New Jersey	170 (49)	74 (20)	0 (0)	N/A (N/A)
New Mexico	54 (22)	74 (56)	6 (9)	.00 (.22)
New York	45 (36)	56 (51)	16 (19)	.25 (.32)
North Carolina	18 (18)	0 (0)	0 (0)	N/A (N/A)
North Dakota	42 (19)	27 (13)	8 (11)	.13 (.18)
Ohio	43 (31)	31 (26)	3 (8)	.00 (.38)
Oklahoma	8 (6)	26 (25)	0 (1)	N/A (1.00)
Oregon	63 (5)	0 (0)	0 (0)	N/A (N/A)
Pennsylvania	18 (0)	21 (18)	0 (0)	N/A (N/A)
Rhode Island	46 (18)	30 (22)	2 (5)	.50 (.40)
South Carolina	81 (35)	38 (8)	1 (2)	.00 (.50)
South Dakota	48 (48)	53 (53)	34 (39)	.35 (.38)
Tennessee	86 (24)	25 (18)	1 (2)	.00 (.50)
Texas	37 (22)	33 (32)	10 (16)	.10 (.13)
Utah	73 (46)	30 (28)	5 (9)	.20 (.11)
Vermont	66 (63)	0 (0)	0 (0)	N/A (N/A)
Virginia	27 (19)	46 (28)	2 (4)	.00 (.25)
Washington	36 (21)	0 (0)	0 (0)	N/A (N/A)
West Virginia	91 (79)	48 (36)	11 (11)	.09 (.09)
Wisconsin	29 (22)	43 (36)	4 (7)	.25 (.14)
Wyoming	44 (22)	20 (20)	1 (2)	1.00 (1.00)

Across all the states, there was a total of 48 stringent, linked, successful objectives and 76 loose, linked, successful objectives. Grouped into functional categories, youth tobacco use had the highest number of successes, followed by patient care/centers and colorectal incidence and mortality for the stringent objectives. For loose objectives, youth tobacco use also had the highest number, followed by vaccination, mammogram use, and patient care/centers tied for third (Table 3).

Table 3.

Number of Successful Objectives by Category.

Category	# Stringent	# Loose
Youth tobacco use	8	10
Patient care/centers	7	7
Colorectal incidence and mortality	6	6
Other prevention	4	6
Breast incidence and mortality	4	6
Lung incidence and mortality	4	4
Vaccination	3	8
Cervical incidence and mortality	3	3
Nutrition and physical activity	2	5
Insurance	2	2
Adult tobacco use	1	5
Mammograms	1	7
Colorectal screening	1	4
Prostate incidence and mortality	1	1
Oral incidence and mortality	1	1
Cervical screening	0	1

The mean stringent success rate was 20% (SD = .22). The mean loose success rate was 28% (SD = .23). Success rates varied by state from 0 to 100% (Figure 1). Other than Wyoming, which had a single linked stringent objective and two linked loose objectives, no state had a success rate greater than 50%.

Figure 1.

Map of success rates (A) Stringent objectives and (B) loose objectives. For both, Alaska has zero successful objectives and Hawaii had missing objectives.

The mean number of total objectives was significantly higher in Plan Past compared to Plan Current (55.06 vs 32.66, t-value = 3.29, P = .0014), while the number of measurable objectives was lower, though not significantly (24.00 vs 30.40). The proportion of measurable objectives (measurable objectives/total objectives) was significantly lower in Plan Past compared to Plan Current (.577 vs .730, t-value = −3.43, P = .0010).

The magnitude of difference between an objective’s baseline and target measurement values did not predict success for stringent objectives. However, it did for loose objectives (F = 4.51, P = .0347). As the magnitude of difference increased, the probability of success decreased (Figure 2).

Figure 2.

Logistic regression results for loose objectives. Objective success was coded as 1 or 0, and the change in objective was the difference between the baseline measurement and the target measurement. Only objectives with percentage measurements were included in the model.

The proportion of successful objectives was not significantly associated with the number of objectives for Plan Past, the number of objectives for Plan Current, or the change in objectives across plans. Furthermore, neither stringent nor loose proportions of successful objectives were significantly associated with the number of linked objectives. Finally, none of the other potential demographic or geographic factors (log of state population size, region, urban percentage, rural–urban status) was significant in predicting objective success rates.

Discussion

Across states, the mean stringent success rate for cancer control plans was under 25%, while the mean loose success rate was under 30%. Unfortunately, 34% of states either had no comparable objectives across plans or did not have two plans to compare, so not all states could be represented. By using logistic regression, I found that one of my suspected underlying reasons—a large difference between the baseline and target —appeared to be a plausible contributor; loose objectives had lower success rate probabilities with higher differences between the baseline and target. My other suspected reasons—too many objectives, and little overlap between objectives—showed no significant contribution. Furthermore, there did not appear to be a difference in success rate across population size, rural–urban status, or region.

After looking at the results, some general comments can be made on what makes a state successful in its cancer control objectives. For this, I will consider the number of successful objectives rather than the success rates, as the actual number of successful objectives is more useful in analyzing successes when looked at individual states as opposed to the overall average (proportion is a better unifying metric that does not depend on the number of objectives). For example, Wyoming had the highest success rate but it only had one stringent linked objective. Similarly, the next highest states—Rhode Island and Nevada—only had two.

South Dakota had the highest number of stringent successes (n = 12), followed by Iowa (n = 5). Arkansas and New York had 4; Kentucky and Idaho had 3; and Missouri, Kansas, and Alabama had 2, while the rest of the states had 1 or less. Correspondingly, South Dakota had the highest number of linked objectives (n = 34). Its number of total objectives and measurable objectives was the same, both for Plan 1 (n = 48) and Plan 2 (n = 53). Similarly, Iowa had the third-highest number of linked objectives (n = 17). Its number of total objectives and measurable objectives was the same for Plan 1 (n = 67) and similar for Plan 2 (n = 45, 40). Therefore, a successful tactic for cancer control plan creators to use appears to be including clearly measurable objectives that remain consistent across plans.

This study had several caveats. First, while each cancer control plan typically followed a similar format, each was unique and required manual entry and decision making that, despite a consistent and rigorous approach as possible, still required personal judgment. None of the numerical measurements, however, were a matter of subjectivity. Second, because over a third of states either did not have multiple cancer plans, did not have measurable objectives, or did not have linked objectives, I was not able to include them in the analysis. Third, there are many other reasons that an objective may not have been met that I could not address in this analysis. For example, states differ in the amounts of money available to implement objectives, number and type of hospitals, approaches to educating the public, county-level metrics, etc.

Fourth, I only used objectives that matched across Plan Past and Plan Current. There were other measurable objectives in Plan Past that could be theoretically determined as successful or non-successful by using survey data like the Behavioral Risk Factor Surveillance System (BRFSS) or a state-level equivalent. Such data collection went beyond the resources I had available in this study. A future study that considers all such measurements could be used to determine if the low success rates I calculated here are valid for non-linked objectives as well. If the success rates were to be significantly higher using that additional data collection, my conclusions would be limited to success rates for only the objectives that had been carried over between plans. However, I believe that my results should match well with the unobserved set of measurement results as objectives that carry through from Plan Past to Plan Current are presumably of high importance, thus necessitating a concerted effort to reach the target. Conversely, it is certainly possible that non-linked objectives could have higher success rates because once objectives are met, they may be dropped from later plans.

The low success rates I found here was surprising given that many cancer control plan reports stress the need for realistic and measurable objectives with planned evaluation.[21-26] While the role of evaluation in cancer control plans is growing,[27] measuring objective success does not appear to be part of standard evaluations. For example, the CDC requires an evaluation plan from all their grantees that includes an indicator of plan objective implementation in data collection methods, but there is no requirement or suggestion on measuring objective success.[28]

For future cancer control plan drafters, I have some suggestions based on the results of this study. First, to ensure a successful cancer control plan, I suggest implementing a measurable outcome for every objective. Second, for each objective, I also suggest carefully selecting the target of the objective to be within an achievable level. A modest but realistic target seems a more sensible approach than an impressive but unrealistic target. To support this suggestion, I compared the success rate between ambitious (top 25th percentile difference between baseline and target) and non-ambitious (bottom 25th percentile difference between baseline and target) objectives and found that non-ambitious objectives, compared to ambitious objectives, were twice as successful for loose objectives (23% vs 11%) and three times as successful for stringent objectives (19% vs 8%). The analyses were not statistically significant, but the trend is supportive of such a suggestion.

Third, while the number of objectives did not appear to be related to success rate, selecting fewer objectives may be beneficial. Focusing on fewer, more important objectives may provide better health outcomes that than many, variously important objectives. Indeed, some of this advice has already been suggested. In a study of three cancer control plan revisions,[29] two of the key factors for successful revision were feasibility and prioritization. Finally, in the evaluation portion of the cancer control plan, the objective success rate should be measured.

Conclusion

Much remains to be done on assessing the success of cancer control plan objectives. As noted earlier, survey data could be used in a future study to capture more information on objective success rate statuses. In addition, a future study could test in detail if the general category of the objective (prevention, detection, survivorship, etc.) predicted success rate. In the longer term, many of the current state cancer control plans only run through 2020 or 2021 and are thus due for renewal. In the next few years, as such plans are made public, they would be ideal targets for an updated analysis of objective success rate. The analysis would build on the initial lessons learned in this report to implement a more systematic and standardized process a priori rather than a posteriori. Furthermore, this initial approach could be refined for use in national and regional cancer control plans outside the United States. Collaboration to develop international guidelines for accessing cancer control objective success rates is a worthy goal and would improve the robustness of these findings.

Cancer control is a difficult problem that requires a well-constructed, realistic, and relevant plan. To date, states have made great strides in constructing and implementing such plans. As states continue to update and improve such plans, learning from earlier iterations will help future constructions and demonstrate evidence-based research both in action and design.

Click here for additional data file.

Supplemental Material, sj-pdf-1-ccx-10.1177_10732748211041504 for Success Rates for the Objectives of US State Cancer Control Plans: A First Look by Mark R. Williamson in Cancer Control