Technical Assistance and Training Needs of Comprehensive Cancer Control Programs: a Qualitative Analysis.

The National Comprehensive Cancer Control Program (NCCCP) was established in 1998 by the Centers for Disease Control and Prevention (CDC) to advance national cancer control implementation across US states and affiliated tribes and territories. To build capacity of NCCCP recipients, technical assistance and training (TAT) is offered in the form of online trainings, webinars, toolkits, workshops, tip sheets, and other products. To determine TAT needs of NCCCP recipients, the George Washington University (GW) Cancer Center conducted a qualitative evaluation to inform TAT planning and implementation. Data on the utilization, applicability, impact, and dissemination of TAT received were collected from comprehensive cancer control practitioners through semi-structured interviews. Detailed memos of interviewee responses were documented and deductively coded based on three themes: promotion of TAT, use of existing TAT, and recommendations for future TAT. Interviewees reported a need for diverse topics, modalities, and TAT reminders. The most widely used TAT resources were social media toolkits, webinars, newsletters, patient navigation resources, and online trainings. Recommendations for future TAT included a focus on coalition support, adaptation and evaluation of evidence-based cancer control strategies, and health equity. Offering a blend of TAT, including educational webinars and trainings, was preferred by CCC professionals and could increase use. Future TAT will provide new opportunities for coalition capacity building, adaptation of evidence-based strategies for cancer control, and center health equity.

Introduction

Cancer remains the second leading cause of death in the USA [1]. Despite steadily declining cancer mortality rates, the American Cancer Society (ACS) expects that almost 1.9 million people will be diagnosed with cancer and 608,570 Americans will die of cancer in 2021 [2]. As of January 1, 2019, there were more than 16.9 million cancer survivors in the USA, a population anticipated to grow to more than 22.1 million by 2030 [3].

Established in 1998 by the Centers for Disease Control and Prevention (CDC), the National Comprehensive Cancer Control Program (NCCCP) was created to promote a collaborative approach for reducing the burden of cancer in the USA through evidence-based cancer control strategies [4]. With more than 20 years’ history of building successful partnerships to address cancer control and improve cancer survivors’ quality of life, the NCCCP currently supports Comprehensive Cancer Control (CCC) planning, implementation, and evaluation in all 50 states, the District of Columbia, eight tribes or tribal organizations, and seven Pacific Island Jurisdictions and US territories. Cancer control plans are guided by six NCCCP priorities: primary prevention, early detection and treatment, supporting survivors and caregivers, and the cross-cutting areas of policy, systems and environmental change approaches, health equity, and evaluation [5].CDC provides CCC programs with funding, guidance, and technical assistance and training (TAT) to develop cancer control plans and also supports affiliated cancer coalitions to further build CCC capacity [6]. TAT, as shown in Table 1, is offered in multiple forms, including online trainings, webinars, toolkits, workshops, tip sheets, and other products. TAT products, designed for CCC professionals and for the practitioners in which they collaborate, aim to improve their collective confidence in advancing evidence-based strategies across the six NCCCP priorities.

Table 1

Forms of TAT products with descriptions

Forms of TAT products	Description
Social media toolkits	Package of pre-written social media messages to assist CCC programs, coalitions, and other professionals in their evidence-informed communication efforts
Webinars	Virtual presentations and networking opportunities featuring subject matter experts to provide timely evidence-based information for CCC professionals
Electronic newsletters	Monthly email newsletters to inform CCC professionals and their partners of new webinars, trainings, events, resources and funding opportunities
Patient navigation resources	A collection of resources for both CCC professionals and their practitioner partners to guide their approach to patient navigation
Online trainings	An online library of eight free asynchronous trainings for CCC professionals and their practitioner partners
Resource repository	Online searchable database of resources to assist CCC professionals

In October 2018, the George Washington University (GW) Cancer Center and American Cancer Society (ACS) each began 5-year cooperative agreements with CDC to provide comprehensive, high-quality TAT to NCCCP awardees and their partners. The objective of the TAT awards is to enhance NCCCP grantees’ ability to implement evidence-based interventions and promising practices in primary prevention, cancer screening and diagnostic follow-up, and cancer survivorship. Research has found that capacity building interventions can significantly improve adoption and implementation of evidence-based cancer control interventions [7]. To be effective, capacity building interventions must involve action at multiple levels, which includes increasing practitioner knowledge and skills, organizational support, and collaboration across organizations in a community [8]. CCC TAT aims to address all three of these levels primarily through the use of education in the form of social media toolkits, webinars, electronic newsletters, patient navigation resources, online trainings, and peer networking.

While the need for capacity building in cancer control is broadly acknowledged, there are relatively few published evaluations of TAT effectiveness [9, 10]. The GW Cancer Center and ACS conducted a structured assessment of CCC program needs by NCCCP priority area, triangulating primary and secondary qualitative and quantitative data collected or published from November 2016 through February 2019 to assess CCC needs. Data came from recent peer-reviewed literature; publicly available and internal evaluation reports and meeting summaries; interviews with cancer control professionals; and email requests to CDC’s NCCCP email listserv. GW Cancer Center staff also routinely solicited information on TAT needs and preferred delivery format from members of its CCC TAT project steering committee, which is composed of CCC professionals. This study reports qualitative findings of the semi-structured telephone interviews that were conducted with CCC awardees and their partners as part of this mixed methods assessment.

Methods

Participant Recruitment

Interviewees were recruited using a master list of CCC program and coalition staff. CCC programs and contact names were provided by the Comprehensive Cancer Control National Partnership (CCCNP) or obtained from program websites. These were entered into an Excel spreadsheet (n = 154).1 Three programs at a time were selected for interviews, using a Google random number generator. Upon selection, GW Cancer Center staff sent an email to the identified contact(s), explaining the nature of the assessment and requesting a telephone interview. If an affirmative response was received, staff scheduled a 1-h telephone interview. If a negative response was received, no further attempts were made to schedule an interview. If no response was received to the original message, staff sent a follow-up email after 2 weeks. No subsequent attempts were made after the first follow-up.

Data Collection

The authors developed a semi-structured interview guide that was edited throughout the interview period as needed to better meet the objective of the assessment. Evaluation questions asked about utilization, applicability, impact, and dissemination of TAT received. Participants shared their prior experience with the TAT provided by the GW Cancer Center and their perception of the impact of GW’s TAT on their cancer control work. Participants were also asked to provide suggestions for future TAT resources or services that would help them implement evidence-based cancer control strategies and advance their cancer plan objectives. Interviewees were not instructed to review the TAT material in advance of the interviews. Staff of 64 CCC programs and/or coalitions were invited, and 50 interviews were conducted with 58 interviewees between July 2017 and September 2019. Of the interviews, 40 interviews were with state programs, four were programs in US affiliated territories, and six were from tribal organizations. Forty-four interviewees represented program directors, managers, and coordinators; nine interviews served as coalition chairs or co-chairs; and five identified themselves in an “other” role. As a CCCTAT provider, the GW staff interviewers were often familiar to interviewees prior to the interviews. Each interview was recorded using Meeting One recording software.2 Detailed notes were documented based on the recordings, and member checking was conducted with interviewees for quality assurance.

Data Analysis

Two authors (RB and MK) deductively coded the data using NVivo 12.0 software [11]. The coders developed a code book that included definitions of three a priori codes: promotion of TAT, use of existing TAT, and recommendations for new TAT. The study team met regularly to ensure consensus in coding. Only explicitly stated (not implied) TAT needs were coded. Repeated concepts were coded only once per interview.

Results

Promotion of TAT

Interviewees indicated trouble identifying appropriate resources when needed due to the large number of resources available. To better assist in identification and utilization of TAT, 43 respondents offered suggestions on how to improve the promotion of resources. Eighteen (36%) recommended providing information and examples of TAT offerings, including one-on-one calls and frequent reminders of available resources. For example, one state cancer control staff member asked, “I don't know if you all have the bandwidth… Could you have calls with everyone to tell them about all your tools?” Another state cancer control staff member said: “[A]s you’re sending out information to promote your resources, every now and then, remind us what you [already] have available.… those reminders are very helpful.” Ten interviewees (20%) also suggested improving the TAT website [12] (www.cancercontroltap.org) to make resources easier to find and accessible. A territory cancer control staff member expressed that: “[I]t’s kind of hard to navigate throughout the site [rather] than just to post a topic on Google.” Another state cancer control staff member agreed:

The only thing I can think of is the resource repository. There is so much in there, but the search functionality isn’t super easy to use. You have to narrow down your criteria so much to get the info you want.

Overall, interviewees expressed the need for a variety of dissemination channels to remind users of available TAT resources.

Use of Existing TAT

Interviewees were asked to identify what TAT resources were most commonly used and most useful. The top five resources, in order of those most frequently identified, included social media toolkits, webinars, electronic newsletters, patient navigation resources, and online trainings.

Social media toolkits are developed by the GW Cancer Center for cancer awareness months to help CCC programs, coalitions, and other professionals strengthen their communication efforts. The toolkits each contain evidence-informed communication strategies, pre-written Tweets and Facebook posts, and other social media and cancer awareness–related tools and resources. Thirty-five interviewees (70%) indicated social media toolkits were useful in their work. A state coalition staff member said:

We also use your messaging from the toolkits and place them on our Facebook page and Twitter all through the month. We just use that to schedule all of our posts. That way, we don’t have to try to come up with different posts. We just use what you have.

Some respondents adapted messaging for specific audiences. A territory cancer control staff member explained: “Sometimes we look at [the social media toolkits] to see the national [information], and then I would try to… adapt it to the local level.”

Webinars presented by subject matter experts share timely evidence-based information for CCC programs and coalitions to apply in their activities. Presented virtually and often recorded, webinars are accessible to both CCC programs and CCC coalition members, and 32 interviewees (64%) indicated they accessed webinars. A state cancer control staff member stated, “We share the webinars with all of our coalition members, if it’s of interest, as well as using them for my own professional development.” Another state cancer control staff member appreciated being able to access webinars on demand: “[I]t’s helpful that your webinars are archived so I can access them at a later date, and I love that the slides are available for download.”

The GW Cancer Center cancer control TAT team publishes two monthly newsletters to inform subscribers of new webinars, trainings, events, resources, and funding opportunities. The Cancer Control Technical Assistance Periodical (TAP) E-Newsletter aims to coordinate and aggregate TAT efforts across multiple national cancer control entities. The Patient Navigation and Survivorship E-News provides information about events, resources, and news related to patient navigation and survivorship for frontline healthcare professionals. Twenty-six interviewees (52%) indicated they regularly used these newsletters to stay informed of CCC related news and resources, including one state cancer control staff member who explained:

Regarding the newsletters: I usually skim them and then if there’s relevant information, I forward it to our partners. They’re always helpful to get an update on what new tools and resources are available.

Twenty-one interviewees (42%) indicated a general confidence in the GW Cancer Center’s expertise in patient navigation and survivorship. A state cancer control staff member mentioned “Everything that I’ve seen that you’ve put out has been just top quality: the webinars and [the materials] around patient navigation and survivorship.”

One resource, mentioned by a state cancer control staff member, was the Implementing the Commission on Cancer Standard 3.1 Patient Navigation Process: A Road Map for Comprehensive Cancer Control Professionals and Cancer Program Administrators [13]:

This one in particular—the Roadmap—I got some really good feedback from [one of our partners]. She does a lot of patient navigation—she’s in distress screening. I got some really good feedback. She said that this is an excellent tool. She was really excited. I don’t always get that level of feedback.

The Patient Navigation Barriers and Outcomes Tool (PN-BOT) [14] was also referenced by a tribal cancer control leader as “a great free service, especially for groups out there that have patient navigation.” PN-BOT is a free data entry, data management, and reporting product designed for oncology patient navigation programs.

The Online Academy offers eight free asynchronous online trainings for healthcare professionals. Twenty-one interviewees (42%) indicated the Online Academy trainings were used by staff and their partners, including a state cancer control staff member who explained:

Another thing that you offer that’s nice are the online academy trainings that are a little more intensive but allow people to do them on their own. We’ve definitely utilized those whenever we’ve had new staff or new contractors, with both the communications and the patient navigation.

Three state cancer control staff members also indicated promoting the Cancer Survivorship E-Learning Series for Primary Care Providers [15]: “We’re constantly promoting the e-learning series, specifically the online Survivorship [E-]Learning Series for Primary Care Providers.” However, many interviewees also discussed challenges in getting primary care providers to complete the extensive training given competing demands for time and continuing education.

Recommendations for New TAT

Interviewees provided insight on topics and formats to deliver resources to further support their local efforts. Support in the areas of CCC coalition building and sustainability, health equity, and evidence-based interventions were the three most commonly referenced topics.

Twenty-three interviewees (46%) indicated that TA on coalition building, development, and sustainability would be helpful. Interviewees, including one state cancer control staff member, indicated experiencing challenges in motivating members to volunteer their time and maintaining the coalition’s self-sufficiency:

The other thing is – and this is very much specific to comprehensive cancer control – how to get our coalition to be self-sufficient and sustainable, diversify funding streams, so they’re not as limited by us and our CDC funding.

Another state cancer control staff member emphasized the need for support to engage and retain coalition members:

I think member retention and recruiting new members and engagement is something that our coalition has kind of struggled with. It’s certainly something that I’ve heard from managers of other coalitions…. It seems like it’s an issue across the board when you’re getting volunteers to come together to donate their time on something.

The need for coalition support spanned across all state, territory, and tribal interviews.

Information on health equity was identified in 14 interviews (28%) as a need for future TAT. Interviewees acknowledged that while some TAT on health equity is available, additional health equity assistance would be helpful, specifically on capacity building as well as targeted TAT for additional audience types. A state cancer control staff member shared:

I think it would be great to start to revisit health equity TA opportunities. I know that GW has TA around health equity...I think most of it is primary care focused or clinical focused, but I think it’d be useful to have TA opportunities around how to better define health inequities or help equity populations, and best practices on how to reach them programmatically, not just through healthcare programs.

Another state cancer control staff member indicated a need for TAT to help advance equity for specific populations: “I don’t think we would turn away from any TA that was offered with either disparate populations like the LGBT communities or the rural [population].”

Interviewees described familiarity with what and where to access information on evidence-based interventions, but further assistance in adapting, implementing, and evaluating interventions was identified by 11 interviewees (22%) as a need, along with examples from the field. A state cancer control staff member recommended additional support in identifying evidence-based interventions to address quality of life needs:

While it’s easy for us to adapt evidence-based interventions around prevention and early detection, when we get into treatment and survivorship quality of life, there really isn’t a strong evidence-based repository for that kind of work. So, implementation guides and best practices in those fields is what I see as a big gap and something that you probably would be well aligned to take on.

Another state cancer control staff interviewee noted the need for evaluation TAT, as well: “A webinar on how people actually go about doing interventions, the process of developing it, and the process of evaluating it, I think, would be fascinating.”

In addition to specific TAT topics, interviewees commonly referenced four delivery modalities: webinars (25), in-person opportunities (20), downloadable tools or templates (16), and examples from other CCC programs and coalitions (15). When discussing the modality of delivering TAT, interviewees also acknowledged there are limitations to many and that a combination of TAT offerings by modality would best suit their needs [see Fig. 1]. A state cancer control staff member explained:

Fig. 1

Frequency of recommended TAT by delivery format. *Other resources include resources recommended 4 or fewer times

What would work best for me would be a combination of webinar and face-to-face…. For example, face-to-face in a small group with follow-up webinars…. The combination is much more helpful for me.

Another state cancer control interviewee appreciated printable resources:

PDFs or resources that are published online that we can easily print and disseminate and share – those often get used more than some of the interactive online resources where you have to find the information over and over again.

Appreciation for peer-to-peer learning was also a theme, explained here by a state cancer control staff member:

Getting to see real-world examples – especially from states that are similar in demographics, geography, or funding – would be helpful. In my opinion, each state operates in its own vacuum. Even though we’re all working on the same CDC-funded work plan… there are thousands of different approaches to the same cooperative agreement.

Discussion

Despite a substantial investment in cancer control programs by the CDC, there are few published evaluations of comprehensive cancer control TAT [9, 10]. Extant evaluations include impact of an online continuing education series to build capacity in cancer survivorship care among clinicians [9] and the impact of workshops on improving cancer control professional capacity to implement human papilloma virus (HPV) vaccination and colorectal cancer (CRC) screening initiatives [10]. Other literature has described the broad impact of communication campaigns in cancer control [16] and CDC investments in training field epidemiologists in low- and middle-income countries [17]. To date, no known studies have described the technical assistance needs of CCC programs and coalitions, which is vital to inform future TAT funding support and approaches.

Primary data collected by interviews validated information gleaned from secondary data sources that were collected and reviewed as part of the larger GW-ACS needs assessment, which included a review of recent peer-reviewed literature, publicly available and internal evaluation reports and meeting summaries, and qualitative data collected through interviews with cancer control professionals and from email requests to CDC’s NCCCP email listserv. The addition of interviews offered a unique opportunity to strengthen relationships between CCC practitioners and TAT providers, obtain more nuanced information from CCC practitioners, and offer customized reminders of TAT products.

Both the interviews and secondary data review identified coalition building and sustainability, health equity, and evidence-based interventions emerged as areas of greatest need for TAT among CCC practitioners. Mention of these topics became more prominent after the 1701 NCCCP cooperative agreement introduced new requirements of CCC programs including salary spending limits, health equity emphasis, and activities limited primarily to evidence-based intervention implementation. The timeline of interviews provided unanticipated insight on the importance of TAT for CCC programs and coalitions to evolve with changing programmatic structures.

Interviews also confirmed that the best approach to TAT is a combination of TAT offered through various modalities, as suggested by Leeman et al.’s (2015) revised framework for capacity building interventions [9]. Based on the framework, TAT delivery can come in variety of forms due to needs and contextual factors, as best explained by a state cancer control staff interviewee: “It’s nice to have it offered in different formats so you can take advantage of it, depending on my needs and my [availability].” Education TAT offerings, including webinars and online trainings, should continue to be utilized as part of a comprehensive set of TAT tools to address the evolving needs of CCC practitioners.

Limitations and Strengths

While the interviews provide helpful information to those offering TAT to CCC programs and coalitions, some caution should be taken when interpreting findings. Given the pre-existing relationship between interviewers and interviewees, social desirability bias may have been present in responding favorably to existing TAT offerings. Additionally, data was collected and analyzed over an extended period of time, and TAT has evolved to address needs found in the needs assessment. Thus, the type and degree of need identified in this assessment likely changed over time and continue to evolve. It should be noted that this evaluation concluded in early 2019, prior to the COVID-19 pandemic. Future needs assessments will explore the impact of COVID-19 on CCC implementation and practitioners’ TAT needs, including effective approaches to virtually deliver TAT. Despite these limitations, these qualitative data offer robust detail to assist in developing and disseminating effective TAT.

Future Directions

Upon the completion of the interviews, TAT has been adapted to incorporate assessment findings, including the development of virtual educational workshops and collaborative CCCNP quarterly coalition webinars focused on health equity. Future TAT will be developed in a variety of formats to fit the unique needs and context of CCC practitioners. Education, through the form of webinars and online trainings, will remain a fundamental component of TAT offerings and will respond to the needs reported in this assessment by expanding capacity in health equity, coalition building and sustainability, and the adaptation and evaluation of evidence-based interventions. A new update to the internationally accessed Oncology Patient Navigation Training: The Fundamentals has been released along with customizable power-point slide decks to allow local practitioners to adapt content to their needs [18]. Dissemination channels are currently being refined to address reported needs for a better online user experience and additional reminders of existing TAT.

Conclusion

Our findings provide guidance for cancer control TAT providers. Education remains a critical component to TAT and capacity building. Offering education in a variety of formats allows for different learner types to utilize resources at the time and place in which they find beneficial. Ongoing evaluation can assist TAT providers in determining the needs of practitioners and improve the effectiveness of TAT in addressing those needs. When possible, the addition of qualitative data through interviews offers a unique opportunity to understand the context in which TAT is needed and how it may best be adapted to address those needs.