Lessons Learned From the Creating Active Communities and Healthy Environments Toolkit Pilot: A Qualitative Study.

The US Army Public Health Center developed the Creating Active Communities and Healthy Environments (CACHE) Toolkit to help military installations evaluate the quality of their built environments relative to healthy eating, physical activity, and tobacco-free living. This study sought to improve its implementation process and assess subsequent Action Plan Guides' utility at 5 military installations. Baseline data included a knowledge, attitudes, and beliefs survey (N = 34); post-Toolkit implementation data included focus groups (N = 2) and interviews (N = 10). Although >80% of participants agreed the built environment affects healthy living, only 44%, 53%, and 35% agreed their installations' built environments promoted healthy eating, physical activity, and tobacco-free living, respectively. Emerging themes comprised "Opportunities to Improve Toolkit and Action Plan Guide Functionality," the "Sociopolitical Landscape Affects Toolkit Implementation," and the "Sociopolitical and Physical Landscapes Affect the Toolkit's Value and Utility." This study provides concrete lessons for the CACHE Toolkit and other public health-based military initiatives.

Introduction

Chronic diseases and conditions such as heart disease, stroke, cancer, diabetes, obesity, and arthritis are common and preventable health problems. An analysis of 2012 National Health Interview Survey data indicated 49.8% of US adults had 1 or more chronic medical conditions.[1] In addition, according to the Centers for Disease Control and Prevention (CDC), chronic diseases were 7 of the top 10 causes of death in 2016 and their treatment accounted for 86% of all health care costs.[2] Chronic disease conditions are burdensome to the military, both in economic cost and military performance. Regarding the latter, they can hinder the ability of military personnel to build resilience and demonstrate the readiness needed for successful service.

Although performing adequate physical activity, consuming a healthy diet, and eliminating tobacco use are 3 key behaviors that can help prevent chronic disease incidence,[2-4] they are not widely practiced in the military. Specifically, results from the 2011 Department of Defense (DoD) Health Related Behaviors Survey of Active Duty Military Personnel indicate 63.1% of active duty service members met Healthy People 2020 moderate physical activity recommendations (an average of 150 min/wk), but only 25.9% met vigorous physical activity recommendations (an average of 75 min/wk).[5] Poor diet is of similar concern, as only a small percentage of active duty service members reported eating 3 or more servings/day of fruit (11.2%), vegetables (12.9%), or whole grains (12.7%).[5] Rates of moderate to heavy smoking (18.3%) and smokeless tobacco use (19.8%) also remain high among service members.[5]

Understanding factors that influence physical activity, diet, and tobacco use are of critical importance to military public health, particularly because the military is committed to making improvements in these areas.[6-8] According to the Social-Ecological Framework, multiple intrapersonal, interpersonal, community, environment, and policy-related factors can affect one’s health risk behaviors.[9,10] Although many current military programs intervene at the intrapersonal and interpersonal levels, few have addressed the built environment. The built environment on military installations, defined as the “physical makeup of where we live, learn, work, and play,”[11] is a modifiable aspect of military life that can help support population-wide health behavior change. To address the need for relevant, evidence-based resources for evaluating the relative quality of installations’ built environments, the US Army Public Health Center (APHC) created the Creating Active Communities and Healthy Environments (CACHE) Toolkit.

The goal of the CACHE Toolkit is to aid local leaders in (1) identifying improvement areas, (2) prioritizing community needs, and (3) developing action plans to maximize the promotion of healthy behaviors.[12] Given the importance of using culturally specific environmental tools and processes to accurately capture the built environment of military installations, a study was conducted from September 2014 through July 2015 with the following objectives: (1) to describe potential users’ perceptions of and attitudes toward the built environment on military installations; (2) to understand users’ experiences with using the CACHE Toolkit to assess their installations built environments and identify ways to substantially improve the tools and Action Plan Guides to meet users’ needs; and (3) to identify additional factors that are important to consider when attempting to intervene with a military installation’s built environment. The study focused on evaluating the Toolkit’s implementation rather than any outcomes. This article highlights CACHE Toolkit study findings that can inform other military studies, initiatives, and policies looking to assess and intervene within the installation built environment.

Materials and Methods

The CACHE toolkit and action plan guides

The CACHE Toolkit has 4 components. First, the Military Nutrition Environment Assessment Tool (m-NEAT), adapted from the Nutrition Environment Measures Survey created at the University of Pennsylvania,[13-15] assesses an installation’s environment—including the workplace, public facilities, restaurants, and food stores—and policies toward healthy eating. Second, the Promoting Active Communities (PAC) tool, adapted from the Michigan Department of Community Health PAC,[16] assesses an installation’s environment, policies, and programs related to physical activity. Third, the Quantitative Indicators for Tobacco Systems (QITS) tool, adapted from the CDC Community Healthy Assessment and Group Evaluation tool,[17] assesses an installation’s policies and environment regarding the promotion of tobacco-free living. Finally, the APHC created supporting documents, including presentation templates, factsheets, an Excel spreadsheet, and a facilitator’s guide for the CACHE Toolkit’s implementation. The facilitators received the Toolkit components in PDF and Excel formats. Each tool was first completed via pencil and paper; answers were then entered into Excel sheets that automatically scored their results. Higher scores indicated a more supportive built environment. After each installation completed and submitted the CACHE Toolkit, the APHC analyzed their data to develop installation-specific Action Plan Guides. The Action Plan Guide included scores for each tool in the Toolkit, tool components that received the lowest scores, and specific recommendations for improving the scores.

Study participants

The study included 5 installations. The APHC team (led by Steven Bullock) selected 4 of the 5 study sites based on participation in concurrent and related health initiatives called Operation Kid Fit (OKF) and the Healthy Base Initiative (HBI). Each installation had an OKF facilitator, whose primary roles were to serve as health educators and as points of contact (POCs) in the CACHE Toolkit implementation process. Health promotion staff at a fifth installation volunteered to participate in the study. The APHC team instructed the CACHE Toolkit POCs to develop a CACHE Toolkit coalition to collect data and return the completed Toolkit for review. The APHC team then provided installations with Action Plan Guides with recommended next steps to improve the built environment, which the CACHE Toolkit coalition could discuss, prioritize, and implement locally. An additional goal was for the CACHE Toolkit coalition to provide leadership with updates through semiregular briefings at the Community Health Promotion Councils (CHPC) on Army installations and the Community Action Information Board (CAIB) on Air Force installations. Community Health Promotion Councils and CAIB consist of installation personnel, all of whom have an interest in the public well-being of the installation community and knowledge of installation health policies and resources that would facilitate the CACHE Toolkit’s implementation. These personnel include but are not limited to the installation’s Health Promotion Officer; the Morale, Welfare, and Recreation Director; the sexual assault advisor; and the public health nurse. The makeup of these groups varies slightly, but all report to the senior commander on each installation.

Data collection

The study assessed the implementation of the CACHE Toolkit through both process and program evaluation. The APHC team collected data quantitatively via a survey, as well as qualitatively via focus groups and interviews. Both approaches were developed concurrently and emphasized equally in the study design. The APHC Public Health Review Board approved this project (#14-299) as Public Health Practice (ie, program evaluation) and not research.

First, participants completed a baseline knowledge, attitudes, and beliefs (KAB) survey in Fall 2014. The survey assessed these constructs in relation to the built environment and policies supporting healthy eating, physical activity, and tobacco-free living. Aside from demographic questions, most questions used 5-point Likert-type scale response categories. The APHC team collected all surveys using Vovici® (version 6; Vovici Corporation, Herndon, VA, USA). Installations implemented the CACHE Toolkit through spring 2015. After the CACHE Toolkit implementation, the qualitatively trained APHC team visited each installation between May and June 2015 to conduct semi-structured focus groups and in-depth interviews with CACHE Toolkit facilitators to discuss their experiences with the Toolkit. Installations who received Action Plan Guides prior to the site visits (n = 4) also discussed Action Plan Guide usability and usefulness. One interviewer led each session and 1 note taker recorded it using digital audio recorders. The APHC team developed a semi-structured guide of 20 open-ended questions to facilitate discussion during interviews and modified it as needed for focus groups.

Post-CACHE Toolkit implementation, participants retook the KAB survey to examine any changes in responses. However, as only 8 participants completed the post-survey, this article only focuses on baseline survey results. Although the study’s intended design was to equally weight qualitative and quantitative data, the robust qualitative data were weighted more than the cross-sectional quantitative data in the analysis and results.[18-20] Survey, focus group, and interview questions are provided in Supplementary Appendices A and B.

Statistical analysis

The APHC team transferred raw survey data to Excel files using the survey program Vovici. They transcribed interviews and focus groups verbatim from audio-recordings. To ensure confidentiality, the APHC team de-identified survey data, coded participants and installations alpha-numerically, and redacted all identifying information in transcripts. Per a Data Use Agreement between the APHC and Tufts University to support the analysis, interpretation, and reporting of these evaluation data, the APHC then transmitted the data via secure, password-protected folders to the Tufts University evaluation team (Marissa M Shams-White, Fernando Ona, and Aviva Must) conducting the analyses. As the evaluation team received de-identified data, the Tufts University Institutional Review Board (IRB) deemed the analysis portion of the study exempt from full IRB review.

The evaluation team conducted univariate analyses of the categorical, quantitative data to summarize participants’ demographic information, as well as to develop descriptive summary data on their KAB related to nutrition, physical activity, and tobacco use. All quantitative analyses were conducted using SAS (version 9.4; SAS Institute Inc., Cary, NC, USA).

Qualitative analysis

The evaluation team analyzed qualitative data from focus group and interview transcripts using a hybrid methodological approach.[21] To orient the evaluation team to the data, one coder (Marissa M Shams-White) developed the initial coding schema with deductive thematic coding guided by survey topics and quantitative findings. The quantitative results helped to iteratively guide initial qualitative analyses.[18,22,23] Interview topics and discussions within 4 randomly selected transcripts also guided initial coding. Next, the coder conducted cycles of inductive coding and used axial coding for the second cycle of coding.[24] After coding every second transcript, the coder reviewed all previous transcripts to achieve intra-coder agreement for internal consistency (>85%). A codebook was created during the hybrid coding approach and updated as needed throughout the coding process. A second coder reviewed each round of coding and 2 coders reviewed the codebook to check for consistency across transcripts. Through an iterative, weekly process, the evaluation team categorized codes into metacodes based on their frequency of occurrence, the underlying meaning across codes, and the relationship between codes. This process continued until themes and subthemes emerged. After themes were detailed, emblematic quotes for each subtheme were extracted into table matrices. Qualitative analyses were conducted using NVivo (version 11; QSR International, Ltd, Burlington, MA, USA). Finally, the evaluation team developed summary recommendations across all the subthemes based on participants’ feedback.

Results

A total of 34 participants completed the baseline KAB survey pre-CACHE Toolkit implementation. Characteristics of these participants are detailed in Table 1. Most participants (79.4%) had no formal training in any components of the CACHE Toolkit prior to the study (Table 1). Two interviews were conducted on each of the 5 installations for a total of 10 interviews. The APHC team held focus groups on the 2 installations that successfully formed working groups.

Table 1.

Characteristics at baseline for all participants (N = 34).

	All (N = 34)N (%)
Frequency attend army installation’s CHPC or air force installation’s CAIB meetings
Never	9 (26.5)
Rarely	5 (14.7)
Sometimes	4 (11.8)
Often	3 (8.8)
Always	13 (38.2)
Participate in CACHE working group
Yes	29 (85.3)
No	1 (2.9)
Working group undetermined	4 (11.8)
Received formal training to date on the tools in CACHE
Yes	6 (17.7)
No	27 (79.4)
Not reported	1 (2.9)

Abbreviations: CAIB, Community Action Information Board; CHPC, Community Health Promotion Council.

Quantitative results

More than 80% of survey participants agreed or strongly agreed that the food, physical activity, and tobacco environments in their communities affect their behaviors in those realms (Figure 1A). However, fewer participants agreed their installations’ built environments promoted healthy eating (44%), physical activity (53%), and tobacco-free living (35%) (Figure 1B). Most participants believed they had a strong understanding of how the built environment affects nutrition (79.5%), physical activity (82.3%), and tobacco use (76.5%) (Figure 1C), and that evaluating the built environment can have a positive impact on these aspects of their installations (Figure 1D). However, only approximately half of the participants believed their leadership prioritized improving the built environment (Figure 1E).

Figure 1.

Participants’ beliefs from the knowledge, attitudes, and beliefs survey at baseline—percentage of participants who selected—Likert-type scale responses “Agree” or “Strongly Agree.”a Participants’ beliefs (A) about the built environment’s impact on food, exercise, and tobacco use in their communities; (B) if their installations’ built environments promote healthy eating, physical activity, and tobacco-free living; (C) regarding their understanding of how the built environment impacts nutrition, physical activity, and tobacco use; (D) about the effect that evaluation of their installations’ built environments can have on improving healthy food availability, physical activity opportunities, and tobacco-free living; and (E) regarding their leadership’s priority to improve the built environment for healthy eating, physical activity, and tobacco-free living (N = 34).

aAs opposed to participants who selected “Neutral,” “Agree,” or “Strongly Agree.”

Qualitative results

As previously mentioned, quantitative evidence provided a priori codes that resonated with emergent codes. Three overarching themes emerged from the iterative coding: Toolkit and Action Plan Guide functionality; the sociopolitical landscape affects Toolkit implementation; and the sociopolitical and physical landscapes affect the CACHE Toolkit’s value and utility.

Overarching theme 1: opportunities to improve Toolkit and action plan guide functionality

This theme encompasses the usability of the tools and Action Plan Guides themselves, internal factors affecting the Toolkit, and external factors that influence tool functionality. Most participants believed m-NEAT, PAC, and QITS were all important and well-organized tools to assess their installation. They highlighted the importance of the user-friendly formats (eg, numbering, labeling, and charts to organize Toolkit information) and evidence-based questions. Reported areas of concern are summarized below into 4 subthemes, with key participant quotes presented in Supplementary Table 1.

Subtheme 1

1: the need to address question relevancy.

This subtheme encompasses participants’ perspectives regarding the relevancy of questions in the Toolkit for their installations. Overall, participants expressed the need for tailored questions in all 3 tools based on the context of their installation or for a “non-applicable” response option. Some participants also shared that the nature and complexity of their worksites made it difficult for them to adequately respond to questions. In addition, participants felt questions were not relevant if they addressed areas too difficult to impact at the interviewee’s level.

2: the need for guidance.

Most participants expressed a need for guidance to complete the CACHE Toolkit. However, those who had previous experience with a tool (eg, the Air Force completes m-NEAT yearly) or previously collected some information requested in the tools found implementing the Toolkit to be straightforward and quick. Conversely, those who lacked experience with the tools or were in situations where no data were collected for any similar projects found Toolkit implementation more challenging. The participants who reported struggles with the Toolkit expressed confusion due to the large size of the installations or diversity of buildings, the use of civilian rather than military terms in the Toolkit, and/or the uncertainty of whom to ask to obtain requested information. They expressed that installations would benefit from additional guidance overall if the Toolkit is implemented throughout the military.

3: the need to include subject matter experts.

Many participants expressed the importance of involving subject matter experts (SMEs) during Toolkit implementation. They believed that some portions of the Toolkit required knowledge beyond the Toolkit facilitators’ level of understanding and that SME inclusion ensured the accuracy of collected information. Examples of SMEs provided by participants included registered dieticians to assist with the m-NEAT, community planners or an employee from the safety office to assist with the PAC, and a tobacco cessation nurse to assist with the QITS. Those who used trained SMEs reported that implementing the Toolkits was quick and easy. However, some participants highlighted 3 main barriers to involving SMEs that they experienced and believed may be potential barriers to future installations. First, due to high job turnover, SMEs new to their positions may not have the contextual knowledge to answer some questions in the Toolkit. Second, some SMEs may be unreceptive when contacted by CACHE Toolkit facilitators and disinclined to assist in implementing the Toolkit. Finally, SME’s busy schedules and existing duties impeded most installations from forming CACHE Toolkit coalitions; the successful formation and meeting of coalitions may promote SME participation.

4: the need to address the Action Plan Guide’s formatting and scoring.

Of the 5 installations, 4 received Action Plan Guides from the APHC team and the participants remarked that the overall format of the reports was clear and informative. Moreover, they appreciated the evidence-based information provided in the Action Plan Guides. However, the scoring used in the tools garnered mixed reviews, as some reported scores included in the Action Plan Guide results to be clear and self-explanatory, while others found scores difficult to decipher and recommended providing increased scoring transparency. In addition, a few participants believed the scoring was too unforgiving with unfair penalizations for specific components; they recommended revisiting the strictness of the scoring criteria.

Overarching theme 2: the sociopolitical landscape affects Toolkit implementation

This overarching theme encompasses the social and political interactions and networks on an installation that affected the timely implementation of the CACHE Toolkit. It is described below in 3 subthemes, with key participant quotes presented in Supplementary Table 2.

Subtheme 2

1: installation complexity.

Participants on large installations expressed that collecting data on the whole installation for the Toolkit was daunting and time-consuming at times. In addition, installations with a variety of workers and, as 1 participant coined it, “hodge-podge” worksites (eg, active duty service members of different branches, union workers) can have various policies and viewpoints that may conflict with one another and make answering policy-related questions challenging.

2: leadership and key players’ support.

The degree of leadership support, degree of key player support, extent of leadership prioritization, and the extent of key players’ prioritization all affected Toolkit implementation. Almost all participants emphasized the importance of garnering leadership support to help propagate important information, create environments conducive to change, and promote key players’ support. Once leadership and key players are on board, it is then important to have them prioritize improving the built environment to increase the potential impact of the CACHE Toolkit.

3: leveraging social networks.

Leveraging social networks to build coalitions and collect information aided some participants in collecting data in a timely fashion. As 1 participant summarized, “Most of it is word of mouth and getting people.” However, Toolkit implementation took longer for those who did not leverage social networks, as well as those challenged by shrinking social networks and increased workloads due to position cuts.

Overarching theme 3: sociopolitical and physical landscapes affect the cache Toolkit value and utility

The final theme addresses the sociopolitical interactions, networks, and physical landscape of an installation that affect the feasibility and successful implementation of Action Plan Guide recommendations. In total, 7 subthemes emerged that affected the utility of the CACHE Toolkit as summarized below. Key participant quotes are presented in Supplementary Table 3.

Subtheme 3

1: policies support enforcement.

An important topic that came up in all the interviews and focus groups was how detailed policies can drive impactful changes. Most participants discussed the lack of policies and initiatives to improve the built environment on their installations, as well as the need for mandates from leadership to enforce existing policies.

2: “tobacco is the culture.”

Although the aforementioned subtheme addresses tobacco policies, the pervasiveness of tobacco use on military installations and the frequency of its discussion in interviews should be acknowledged. As 1 participant summarized, “Tobacco is the culture.” Participants described the easy access enlisted Soldiers have to tobacco products, the use of tobacco breaks to form relationships with leaders, and the presence of officers modeling tobacco-promoting behaviors as large barriers to changing tobacco policies on military installations.

3: entities with competing interests.

Participants discussed how the goals of the CACHE Toolkit currently conflict with the interests of several entities on installations. These entities may include Army and Air Force Exchange Services (AAFES), food vendors with contracts with the installations, unions, and schools.

4: high vs low traffic food locations.

Although changes can be implemented to improve the food environment, the physical location of food-serving outlets can affect the value of making these changes. Some changes in high-traffic locations, like at dining facilities (DFACs), can positively impact the nutritional choices of service members. Conversely, some installations offer healthier food options in low-traffic locations, which is a waste of resources, or lack food establishments entirely, forcing service members to leave the installation to purchase food, respectively.

5: the landscape for physical activity.

Similar to food-serving locations, the location of physical activity resources affects the value of changes to the physical activity environment. Participants discussed how some walking and hiking paths are hidden due to a lack of signage and how the presence or absence of biking lanes, bike racks, and sidewalks affected the popularity and safety of biking and walking on installations.

6: budget limitations.

As is commonly experienced with many public health interventions, the implementation of many action plan recommendations was impeded by budget limitations. Although some small recommendations were feasible, almost every participant via interviews and focus groups reported many recommendations were too costly. Action Plan Guide recommendations to change the built environment therefore need to take into account the potentially limited finances available to installations.

7: local vs centralized changes.

Given that the goal is to have the tools in the CACHE Toolkit used across the military, and given the differences in policies among Services, participants noted that Action Plan Guide recommendations need to be tailored to the specific military branch of the installation assessed. Participants primarily cited the differences between the Army and Air Force and how they can make changes at the local vs centralized level.

Discussion

The goals of this study were to understand CACHE Toolkit users’ perceptions of and attitudes toward the built environment on their military installations, evaluate the process of implementing the CACHE Toolkit, assess the efficacy of subsequent APHC Action Plan Guides, and identify ways to improve on both for future implementation. This study concludes that with revisions to the tools and process, the CACHE Toolkit can be a valuable resource for military installations.

Survey results highlighted the importance of the CACHE Toolkit, as most participants believed that evaluating installations’ built environments can guide improvements. In addition, although most participants believed installations’ food, physical activity, and tobacco environments affect employees’ healthy eating, physical activity, and tobacco-free living, fewer than half agreed that their installations’ built environments promoted these positive behaviors. As interventions assessing military installations’ built environments are currently limited, those conducting similar initiatives may learn from the experiences from this study.

One important conclusion is Toolkit-specific: respondents highlighted the importance of providing detailed assessment tools and Action Plan Guides to improve their functionality. Specifically, carefully chosen questions with clear, military service-appropriate verbiage; adequate support to facilitators via the APHC, SMEs, and working groups; and transparent scoring of questions in the guides are essential.

Second, participants highlighted the importance of leadership support and their prioritization to improve the built environment to propagate healthy changes. Approximately half of survey respondents believed their leadership prioritized improving the built environment for healthy living. Qualitative findings corroborated these quantitative findings. The overarching theme that the sociopolitical landscape affects Toolkit implementation captured the idea that timely and accurate Toolkit implementation can be impacted by leadership’s support and prioritization. That is, if leadership creates a milieu that encourages positive changes to the built environment, key players, SMEs, and others are more likely to commit their time and effort to working groups and providing timely responses.

The third main conclusion is that higher command must create policies that detail how to make and enforce positive changes to the built environment. Only about one-third of survey participants believed their installations’ built environments promoted tobacco-free living. Qualitative analyses further supported this: many respondents emphasized the lack of enforcement of tobacco-related policies and the barriers to changing the culture surrounding tobacco use on the installations. Smith and Malone[25] also examined the barriers in the military to change tobacco controls, and similar to our findings, highlighted the tobacco culture, lack of policy enforcement, and the tobacco rights of civilian personnel on installations. Participants in our study expressed that both policy interventions from DoD level command and policy enforcement from installation-level command represent the only ways to impact the “tobacco culture” on military installations. Smith and Malone[25] also highlight the need for updated regulations, despite inevitable pushback. Moreover, policies can also be established to impact relationships with entities with competing interests (ie, AAFES, unions, food contractors, schools).

Fourth, the locale of food and physical activity promoting locations can affect their perceived value. Only 44% and 59% of survey participants agreed or strongly agreed that their environments promoted healthy eating and physical activity, respectively. Qualitative findings highlighted that making healthy food changes to Shoppettes (ie, installation convenience stores), vending machines, and other food serving facilities will have less impact in low-traffic locations. Similarly, employees must be aware of the presence of walking trails and safe areas with sidewalks and bike lanes to increase foot and bike traffic.

Finally, financial, service-specific, and installation-specific limitations require consideration in Action Plan Guide recommendations. Due to budget constraints, smaller, less costly recommendations are more likely to be feasible in the short-term than larger, costly recommendations. The APHC should consider the receptivity and feasibility of recommendations made to individual installations, as well as the readiness and capacity of installations to implement changes.

This study has a few limitations worth noting. First, we were unable to examine changes in KAB following the CACHE Toolkit intervention due to the limited number of participants who completed the post-survey (n = 8). This affected our ability to give greater weight to our quantitative findings. Second, changes in the built environment emerging from the CACHE Toolkit Action Plan Guide recommendations could not be evaluated, as none of the installations had implemented these recommendations prior to the interviews and focus groups. However, the study did provide an opportunity for the APHC to identify key barriers and facilitators to the CACHE Toolkit Action Plan implementation processes and, thus, still offers valuable lessons. Third, 2 of the 5 recruited installations already implemented m-NEAT and PAC as part of overlapping initiatives (eg, HBI), while other CACHE Toolkit facilitators recruited community planners to implement PAC. In both instances, some participants did not have experiences to share regarding the full CACHE Toolkit’s utility. For this reason, rather than comparing the tools’ utility across all recruited installations, the focus was on the process of each tool individually when applicable. Finally, findings from these recruited installations may not be generalizable to other military installations given the range of facilitators’ experiences and variable installation policies and leadership support. However, the goal of this study was not an outcome evaluation or to test a theory, but rather a process evaluation.

This study had many strengths as well. Quantitative and qualitative data were leveraged: the qualitative approach used quantitative findings in initial deductive coding and helped corroborate and elaborate on the survey findings. In addition, though most facilitators did not report previous experiences with any of the CACHE Toolkit tools, some of the installations had legacy tool facilitators (eg, an Air Force Instruction [AFI] requires Air Force installations to implement m-NEAT annually) that provided support and guidance to facilitators when needed. Finally, few studies and initiatives to date examine the built environment on military installations; this study provides concrete feedback to aid the future implementation of the CACHE Toolkit as well as insights for other military public health initiatives.

Recommendations

The recommendations address the aforementioned subthemes and are categorized into short, medium, and long-range recommendations based on the time and effort needed for implementation (Tables 2 to 4). Focus group and interview participants noted specific recommendations to improve the Toolkit and make it more user-friendly which the APHC can immediately address (Table 2). They also emphasized recommendations to guide the CACHE Toolkit and Action Plan implementation processes. These included both obtaining initial leadership buy-in from the start to aid Toolkit implementation and prioritizing Action Plan Guide recommendations that align with the priorities of installation leaders (Table 2). The medium-range recommendations, focused on building up the APHC’s website and funding mechanisms (Table 3), may take more effort and time to accomplish, but can contribute to the future success of the CACHE Toolkit when implemented throughout the military. Finally, long-range recommendations (Table 4) encompassed the importance of creating the “right committee” and the need for DoD to develop policies to support the implementation of both the CACHE Toolkit and Action Plan Guide goals. Regarding the latter, policies should include specific steps on how higher command can execute and enforce the recommended policy changes in an effort to increase engagement in behaviors.[26]

Table 2.

Short-range recommendations: The APHC edits and facilitator guidance.

Recommendations	Details
1. The APHC edits
Create adaptable worksheets	• Add “not applicable” as a response option to questions.• Allow for the skipping of irrelevant questions without scoring penalties.• Add a notes section for response elaboration for when ideal responses to questions are not provided (eg, questions encapsulating “hodge-podge worksites”).
Define terms	• Define key terms to clear up confusion regarding who to ask for information or how to respond to questions (eg, define “healthy option” and “meal”)• Ensure correct, military and branch-specific terms are used for each question (eg, rather than civilian terms or, for Air Force installations, Army terms).
Rethink question inclusion/wording	Reassess questions and remove irrelevant questions• Examine questions in relationship to the establishment (eg, apply to poster base) and determine what are appropriate, meaningful questions.• Put in place a quality assurance (QIQAQC process) mechanism.• Consider removing “higher level questions” that CACHE facilitators have little control over influencing at their level. Alternatively: ○ Retain questions for installations with initiatives/local policies that warrant routine evaluation (ie, assess their utility each time the CACHE is implemented in the future), but include “N/A” as a response and remove scoring penalizations related to them (ie, not skew scoring results for installations choosing “N/A”). ○ For policy questions, specify if questions are directed at the installation or DoD-level to clear up confusion.
Clarify and/or rethink scoring mechanism	• Make scoring weight for each of the included questions transparent.• Consider changing scoring penalizations from set, black-and-white numerical percentages. Suggestions: ○ Progress scores over time. Allow installations to focus on the smaller recommendations first and grant time for bigger changes to be implemented ○ Consider a low, medium and high continuum for scoring instead of percentages to be more translatable.
Develop a more detailed Information Guide	• Supply examples of which SMEs and key players should be contacted to participate and/or provide information for each set of questions. ○ Include tips for effectively communicating with unreceptive contacts. ○ Share more detailed information to guide new facilitators or key players due to job turnover or lack of previous experience with the Toolkit.• Elaborate on Toolkit scoring.
2. CACHE Toolkit Facilitator guidance
Get buy-in from the start	• Get key leaders on installation on board before starting to aid momentum and timely responses from key players and SMEs• Get key players on board (ie, in working group) before starting ○ If they’re invested in the working group, the main facilitator will not have to implement the Toolkit alone, allowing for a timelier implementation of the Toolkit ○ Key players on board can help troubleshoot/decide on optimal paths for tool implementation and share recommendations’ feasibility/best approaches
Segment Toolkit implementation over time	Allow for adequate time to implement the Toolkit (eg, 3-6 months)• Break the Toolkit down into components and make a timeline for implementation• Consider recruiting a contractor who has the time to be the key facilitator and be, in a way, the project manager
Communicate scoring intentions	Make intentions of assessment/scoring clear prior to visiting sites: clear. Send emails, for example, to commanders, schools, community organizations, worksites and building managers, and DFACs explaining what you will be doing.
Be persistent	• As the facilitator, many participants advised to be persistent: “be willing to jump in,” go out and start asking until you can find informants needed.
3. CACHE Toolkit Action Plan Guide Implementation Guidance
“Choose your battles”	Prioritize and “choose your battles” rather than focusing on all recommendations at once.• Choose to focus on areas where commanders may be more invested in and can start making changes• Recognize it is okay to focus on smaller goals, not just the large-scale goals. Go for the “little wins,” for example: ○ Work with AAFES representatives to make vending machine changes ○ Work with the Defense Commissary Agency to post nutrition information ○ Add bike racks for safe bike storage ○ Hold 30 min school education programs (eg, led by external educator to prevent pushback due to teachers’ workloads) and community events

Abbreviations: AAFES, Army and Air Force Exchange Services; APHC, US Army Public Health Center; CACHE, Creating Active Communities and Healthy Environments; DFAC, dining facility; DoD, Department of Defense; N/A, not applicable; QIQAQC, quality improvement, quality assurance, quality control; SMEs, subject matter experts.

Table 4.

Long-range recommendations: Improve the CACHE Toolkit and action plan guide recommendations’ implementation process.

Recommendations	Details
1. Create the right committee
Don’t do it alone	• Recommend against implementing Toolkit alone: break it up into components and include a team with training in key areas to help implement it (see “Include SMEs”)• Include key people who know what is feasible vs unfeasible with Action Plan Guide, as well as who know what is most vs least impactful, for example, include: ○ Those who use and work in built environment (runners/bikers, former tobacco users, spouses involved in installation’s community, staff at DFACs/AAFES) ○ Include SMEs who know limitations of installation (see next recommendation)
Include SMEs	• Based on the assessment information that needs to be gathered, coordinate with SMEs to get access to information that would otherwise be hard to find• Include SMEs in working group to help implement Toolkit, ensure accurate reporting, and ensure suggestions/avenues chosen are feasible ○ For example, community planners, registered dieticians, tobacco cessation nurse, Defense Commissary Agency, DFAC manager• Provide a list of SMEs on installations by branch for facilitators to recruit (eg, to include on the APHC website and/or information guide)
Leverage existing coalitions	• Try to join an existing coalition (ie, working group(s)) with key members already recruited, instead of forming a new one
Turnover: have transition process	• Create transition process for facilitators and working group members (eg, schedule overlap to allow shadowing of new employee)• Provide detailed guides for future CACHE facilitators and working group members ○ Provide details in Information Guide and on website ○ Create log of minutes from meetings/notes from facilitators to pass down to future position holders
2. Policies needed to aid the CACHE Toolkit implementation
DoD-wide policy needed (vs by branch)	• Policies need to be made across branches to allow for healthy built environments across military branches. This should be done: ○ To allow for quicker, routine process evaluations ○ To share successes in 1 branch (eg, AFIs and tobacco-free environments) with other branches• To engage at all levels of the pyramid, not just 1 portion, for example, as happens on joint Air Force-Army installations, since Air Force and Army focus on engagement at the bottom (ie, population level) vs middle and top of the pyramid (ie, one-on-one interventions), respectively.
3. Policies needed to aid the CACHE Toolkit Action Plan Guide implementation
Policy and higher command impact to aid enforcement	• Policies are needed to provide guidelines to execute and enforce DoD’s vaguer, tobacco-related policies• Detailed guides are needed for leadership, building managers, etc. on how to enforce policies to aid employees involved in tobacco cessation in reaching goals
New policy to create new changes	• New policies are needed at installation and especially DoD level to promote:• Physical activity-friendly environments (eg, add sidewalks and bike lanes for safe walking/biking)Tobacco-free environments (eg, negotiations with unions, creating uniform policies on installations)

Abbreviations: AAFES, Army and Air Force Exchange Services; AFIs, Air Force Instruction; CACHE, Creating Active Communities and Healthy Environments; DFACs, dining facilities; DoD, Department of Defense; SMEs, subject matter experts.

Table 3.

Medium-range recommendations: The APHC website and funding.

Recommendations	Details
1. Build on the APHC’s website: to aid the CACHE Toolkit and CACHE Toolkit Action Plan Guide implementations
Online Q&A/FAQs page	Include Q&A and web forum to answer commonly asked questions by installations.• Allows for revised lists of suggestions to be accessible to facilitators as more solutions are strategized by both facilitators and the APHC• For example, SMEs who participated on their respective installations can share experiences/tips, for example with those who may lack access to a SME, those who are new to their position and not fully trained, and/or with facilitators unable to form successful working groups yet.
List of recommended substitutions during roadblocks	• Provide recommended substitutions to roadblocks facilitators and their teams may encounter while collecting information.• Update list of recommendations on a regular basis as solutions are strategized by the APHC/working teams/web forum users• Applies to Toolkit implementation process (eg, unreceptive contacts, unable to locate data) and Action Plan Guide recommendations (eg, substitutions for larger-scale recommendations-such as policy changes, sidewalk and bike lane installations, food offering recommendations)
Online web forum	Include a web forum to help those on large installations clarify areas of concern/connect with others in similar situations to get the help they need (allows for a two-way, timely, dynamic exchange of information, as opposed to one-way sharing by the APHC through the information guide and FAQs page)• Facilitators and SMEs can ask questions and share experiences and tips with one other• Installations with legacy facilitators of tools (eg, Air Force installations and m-NEAT) can provide support/guidance in the forum• The APHC can highlight solutions discovered via the forums on the FAQs and “list of recommended substitutions” pages
Interactive online map and/or app for smartphone	As part of the Action Plan Guide implementation it can help educate users. For example, it can:• Provide an interactive, Google or “Map My Run”-type online map for walking trails and safe walking and biking areas (ie, routes with sidewalks and bike lanes)• Provide nutrition facts and tobacco policy updates for different areas of each installation
2. Address funding limitations for the CACHE Toolkit Action Plan Guide recommendations
• Conditional APHC funding opportunities	Consider offering conditional funding to installations to perform Action Plan Guide recommendations. Require timeline and goals that must be met to secure and retain funding.
• List of recommended substitutions for funding limitations	• Separate recommendations by smaller vs larger recommendations based on time, policy and/or funding; include gradation of recommendations• Offer an alternate path to achieve larger recommendations (ie, how to build on it over time to accomplish larger goal)• Offer a list of recommended substitutions in the Action Plan Guide or the APHC website for less costly interventions• Hold healthy snack bar competitions in the work environments to raise awareness about nutritious foods

Abbreviations: APHC, US Army Public Health Center; CACHE, Creating Active Communities and Healthy Environments; FAQs, frequently asked questions; m-NEAT, Military Nutrition Environment Assessment Tool; Q&A, questions and answers; SMEs, subject matter experts.

Click here for additional data file.

Supplemental material, Supplementary_Material for Lessons Learned From the Creating Active Communities and Healthy Environments Toolkit Pilot: A Qualitative Study by Marissa M Shams-White, Alison Cuccia, Fernando Ona, Steven Bullock, Kenneth Chui, Nicola McKeown and Aviva Must in Environmental Health Insights