Insights from Washington State's COVID-19 Response: A Mixed-Methods Evaluation of WIC Remote Services and Expanded Food Options Using the RE-AIM Framework.

BACKGROUND: In response to the COVID-19 pandemic, Washington State's Supplemental Nutrition Program for Women, Infants, and Children (WA WIC) adopted federal waivers to transition to remote service delivery for certification and education appointments. WA WIC also expanded the approved food list without using federal waivers, adding more than 600 new items to offset challenges participants experienced accessing foods in stores.
OBJECTIVE: This study aimed to assess the reach and effectiveness of the programmatic changes instituted by WA WIC during the COVID-19 pandemic; the processes, facilitators, and challenges involved in their implementation; and considerations for their continuation in the future.
DESIGN: A mixed-methods design, guided by the RE-AIM framework, including virtual, semi-structured focus groups and interviews with WA WIC staff and participants, and quantitative programmatic data from WIC agencies across the state. PARTICIPANTS/
SETTING: This study included data from 52 state and local WIC staff and 40 WIC participants across the state of Washington and from various WA WIC programmatic records (2017-2021). The research team collected data and conducted analyses between January 2021 and August 2021. ANALYSIS: An inductive thematic analysis approach with Dedoose software was used to code qualitative data, generate themes, and interpret qualitative data. Descriptive statistics were calculated for quantitative programmatic data, including total participant count, percent increase and decrease in participation, percent of food benefits redeemed monthly, and appointment completion rates.
RESULTS: All WA WIC participants (n = 125,279 in May 2020) experienced the programmatic changes. Participation increased by 2% from March to December 2020 after WA WIC adopted programmatic changes in response to the COVID-19 pandemic. Certification and nutrition education completion rates increased by 5% and 18% in a comparison of June 2019 with June 2020. Food benefit redemption also increased immediately after the food list was expanded in April 2020. Staff and participants were highly satisfied with remote service delivery, predominantly via the phone, and participants appreciated the expanded food options. Staff and participants want a remote service option to continue and suggested various changes to improve service quality.
CONCLUSIONS: Participation in WIC and appointment completion rates increased after WA WIC implemented service changes in response to the COVID-19 pandemic. Staff and participants were highly satisfied with remote services, and both desire a continued hybrid model of remote and in-person WIC appointments. Some of the suggested changes to WIC, especially the continuation of remote services, would require federal policy change, and others could be implemented under existing federal regulations.

The Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) is a federal program that provides nutritious foods, health screening, nutrition education, breastfeeding support, and referrals to approximately 6.2 million low-income and nutritionally vulnerable women, infants, and children across the United States, including nearly one-half of all infants and 25% of young children. Forty-two percent of WIC participants are Black, Indigenous, and People of Color and the percent of eligible families participating in WIC is higher in rural than urban communities. WIC participation is associated with healthier diets and improved infant morbidity and mortality rates. ,

Research Questions: What did the transition to remote services and expansion of the food list during the COVID-19 pandemic look like for the Washington State Supplemental Nutrition Program for Women, Infants and Children (WA WIC), and how did the changes impact programmatic reach and effectiveness? In addition, do WA WIC staff and participants want these programmatic changes to continue in the future and, if so, how?

Key Findings: Compared with December 2019 (prior to the COVID-19 Pandemic), overall WA WIC program participation was increased in December 2020. In addition, appointment completion rates were increased in June 2020 compared with June 2019. Both staff and participants appreciate the convenience of remote appointments and would like to see a hybrid model combining both in-person and remote appointments continue beyond the COVID-19 pandemic.

Despite program benefits, only approximately one-half of eligible individuals enroll in WIC. The barriers to using WIC services are well-documented and include required physical presence at WIC offices for appointments, limited WIC-approved foods, and negative shopping experiences.7, 8, 9, 10 The COVID-19 pandemic initially exacerbated these barriers, as people were encouraged or required to stay home to limit coronavirus transmission, and grocery stores experienced shortages of WIC-approved foods. , The outbreak also created economic conditions that increased rates and severity of food insecurity among vulnerable families.

In March 2020, Washington State WIC (WA WIC) began offering WIC certifications and other appointments remotely (together referred to as “remote services”), using waivers provided through the Families First Coronavirus Response Act. WA WIC also expanded the list of allowable foods by more than 600 items in April 2020; WA WIC intentionally expanded the list without the use of federal waivers to ensure the expanded access to allowable foods would continue beyond the COVID-19 pandemic. These changes allowed eligible households to participate in WIC when clinics were closed to the public and improved access to WIC foods during retail shortages.

COVID-19 pandemic–related changes to WIC services provide an opportunity to gain insight into improving WIC services over the long-term. This study aimed to assess the reach and effectiveness of the programmatic changes instituted by WA WIC during the COVID-19 pandemic; the processes, facilitators, and challenges involved in their implementation; and considerations for their continuation in the future.

Methods

This mixed-methods study used semi-structured interviews with WIC participants, focus groups and key informant interviews with WIC staff, and participant-related programmatic data to evaluate WA WIC’s transition to remote services and expanded food options during the COVID-19 pandemic. Study protocols were determined exempt from human subjectsreview by the University of Washington Human Subjects Division and Washington State Institutional Review Board.

Framework

The RE-AIM framework (see Figure 1 ) guided the research aims and design, providing a structure for evaluating the programmatic changes’ reach, effectiveness, adoption, implementation, and maintenance. Using both qualitative and quantitative data within the RE-AIM dimensions generated a thorough examination of programmatic changes because qualitative data provided context for quantitative trends.

Figure 1

Constructs of the RE-AIM framework, construct definitions, and data sources for the constructs used to analyze the Washington State Special Supplemental Nutrition Program for Women, Infants, and Children (WA WIC) pandemic response.

Construct	Definition applied in this study	Data source
Reach	The number and proportion of WA WIC participants who experienced the program adaptations	• Programmatic participation data
Effectiveness	The extent to which WA WIC adaptations were associated with maintained or improved:• WIC participation • Appointment completion • Food benefit redemption	• Programmatic participation data • Appointment completion data • Food benefit redemption data • Staff focus groups
Adoption	• Programmatic changes adopted by WA WIC • Reasons for adopting these changes	• Staff focus groups
Implementation	• How the adaptations were implemented • Supports needed to implement the adaptations • Facilitators and challenges to implementation • Satisfaction of WA WIC participants and staff including advantages and disadvantages	• Programmatic data • Staff focus groups • Participant interviews
Maintenance	• Staff and participant interest in maintaining the adaptations • Supports needed to continue or improve the adaptations • Improvements to WA WIC remote services and food list	• Staff focus groups • Participant interviews

Sample and Recruitment

WIC Staff

Convenience sampling was used to recruit state and local WIC staff to participate in focus groups and key informant interviews. Existing contacts within WA WIC identified state-level staff involved with programmatic changes. To recruit local WIC staff, the research team selected 10 of the 57 local WIC agencies from which to recruit staff based on rurality, caseload, region, and service status (eg, if offering curbside services as of May 2020). The lead staff at the state WIC office e-mailed study information to coordinators at each local agency and asked them to share study information with other WIC staff at their agency. The final sample included 52 WIC staff, 10 who worked at the state level and 42 who worked at the local level. In addition to interest in sampling WIC staff from the state agency and a diverse array of local WIC agencies as detailed above, recruitment intentionally focused on staff roles in the program; specifically, the team sought perspectives of staff involved in program coordination, certification, nutrition education, or breastfeeding education at the local level and staff involved in developing the food list and supporting the transition to remote services at the state level. Recruitment continued until invitations of at least 1 staff member in each role were sought from each of the identified agencies. Race and ethnicity, sex or gender, and age information for WIC staff participants were not collected.

WIC Participants

WIC participants were recruited through a banner placed on the WICShopper app in English and Spanish for 1 day, as the WICShopper app was used by approximately 90% of Washington WIC households at the time of recruitment. In that time, 147 WIC participants volunteered by clicking on the banner, which brought them to a webpage with the study procedures and a link to provide contact information and self-report demographic and other key characteristics, including race and ethnicity, length of time on WIC, breastfeeding status, how many (if any) infants and children enrolled in WIC, and the WIC agency they visit. No data on sex and gender or age of WIC participants were collected.

Seventy-two survey respondents were contacted using a maximum variation approach to aim for diverse perspectives of the WA WIC participant population regarding race, ethnicity, rurality, and WIC participant enrollment category (whether they were enrolled, their infant was enrolled, or their child/children were enrolled). The study team made a particular effort to contact participant volunteers identifying as American Indian and Black because these racial groups experience the largest nutrition and infant and maternal health disparities in the state. In addition, the aim was to recruit 10 WIC participants whose primary language was Spanish to ensure that the perspectives of participants who speak a language other than English were included. Spanish is the most commonly spoken language after English in Washington State, and an estimated 6.5% of households receiving cash, food, and medical benefits in the state consider Spanish their primary language. Participant recruitment continued until the goal of 40 total interviewees, 10 of whom spoke Spanish, was reached; the goal was based on prior similar work that indicated 40 interviews would be sufficient to reach data saturation. Of the 72 respondents contacted, 40 completed interviews, 6 declined participation, 11 were scheduled but did not complete interviews, and 15 were unreachable after up to 4 contact attempts.

Data Collection

Programmatic Data

Nonidentifiable programmatic data were provided by WA WIC to examine reach and effectiveness through temporal trends in participation and food benefit redemption. WA WIC also shared reports from local WIC agencies from May and August 2020 with data on WIC service status (eg, number of staff and number of sites offering curbside or face-to-face services), supports needed, and challenges experienced. These data were explored so emergent quantitative trends could inform questions during qualitative data collection and to help triangulate focus group and interview findings.

Staff Focus Groups and Interviews

Ten focus groups were organized to include staff members with similar roles within WIC as able. One focus group included state staff managing the transition to remote services (n = 5), 1 included state staff managing the food list expansion (n = 5), 2 included local program coordinators (n = 7, n = 6), 2 included breastfeeding support staff (n = 3, n = 2), 1 included nutrition educators (n = 5), 1 included certification staff (n = 4), and 2 included staff from a variety of positions (n = 7, n = 6). In addition, 2 local staff participants (program coordinator and nutrition educator) were not able to participate in focus groups due to scheduling constraints, so key informant interviews were conducted with these individuals. The study team developed the semi-structured focus group guide (Figure 2; available at www.jandonline.org) using the study aims and the RE-AIM framework; this guide was also used for the key informant interviews. The questions in the focus group guide asked staff about processes, facilitators, and challenges that accompanied the switch to remote services and the expanded food list. Staff were also asked about the feasibility and support needed to continue aspects of remote services. Two female researchers with masters- or doctoral-level training and experience conducting research in public health and nutrition conducted and recorded approximately 70-minute focus groups via the Zoom platform from January 2021 through February 2021.

Figure 2

Semi-structured focus group and key informant interview guide used with Washington Special Supplemental Nutrition Program for Woman, Infants, and Children (WA WIC) staff to determine their experiences and perceptions of programmatic changes during the COVID-19 pandemic. DOH = Department of Health; RD = registered dietitian.

WIC STAFF INTERVIEW AND FOCUS GROUP QUESTIONS
State Staff–Involved in Changes to Remote Certification and Services
1) Please briefly introduce yourself and describe your role in the WIC program. 2) Federal law currently requires certifications to be conducted in person, but a waiver during COVID allowed this to happen remotely or to postpone certain parts of the certification process. What does remote certification look like in Washington during the COVID pandemic? [Probe: mode (phone, video), and changes in frequency, duration, topics covered] a. To what extent does remote certification look similar or different across the state? If there are differences, what are these and why? 3) Think about how remote certification has been working in Washington during the COVID pandemic. a. What has been working well? b. What has been challenging? What additional supports are needed or logistical challenges need to be addressed? 4) What would you like to see in federal policy about the option for remote certification after the COVID pandemic? Why is that? a. What would DOH and local agencies need to address/clarify to offer remote certification? 5) Next, I’ll walk us through a few WIC services, including nutrition education, health screenings and referrals, and breastfeeding support. I’m interested in hearing from you about how each service has changed since the start of the COVID pandemic, including how and the extent to which these services are now being offered remotely.• How have nutrition education services changed and to what extent are they being offered remotely? How consistent or different are these changes across the state? • How have health screenings and referrals changed and to what extent are they being offered remotely? How consistent or different are these changes across the state? • How has breastfeeding support/peer counseling changed and to what extent are they being offered remotely? How consistent or different are these changes across the state? 6) What resources, training, guidance, or technology has been provided by DOH to support the adoption and implementation of remote services? 7) Think about how remote services (nutrition education, referrals, breastfeeding support) have been working in Washington during the COVID pandemic. a. What has been working well? b. What has been challenging? c. What additional supports are needed? What logistical challenges need to be addressed? 8) Would you like to see remote services offered more, less, or about the same amount after the COVID pandemic? Why? a. If interested in growing or maintaining remote services: i. What supports would be needed to help that happen (eg, technology, training, communications/messaging, other)? ii. What costs would be required? iii. Would any policy changes be required? iv. How feasible or likely does it seem that this could happen? 9) How has offering WIC certification and services remotely impacted service quality, convenience, and outcomes? 10) In addition to the ability to conduct certifications and services remotely, what other WIC changes or flexibilities would you like to see in Washington State to make it easier for clients to use the program during or after the COVID pandemic? a. Are there any priorities that the agency is currently exploring or planning? 11) How have changes to the certification, nutrition education, health screenings and referrals, and breastfeeding support/peer counseling been communicated to WIC clients? 12) We plan to conduct focus groups with local staff and interviews with WIC clients about remote certification, remote services, and the recent expansion of Washington WIC’s list of allowable foods. What would you be most interested in hearing from local staff and clients about these topics?
State Staff–Involved in Changes to Remote Certification and Services
1) Please briefly introduce yourself and describe your role in the WIC program as it relates to the expanded food list. 2) Why did DOH decide to expand the list of WIC allowed foods most recently in April? a. Is the list reviewed and adjusted periodically according to a set timeline or was this a one-time effort based on response to stakeholder feedback or some other reason? b. Since when/for how long had DOH been talking about expanding the list? 3) Please tell me a bit about the process through which DOH decided which foods to add. For example, which stakeholders were involved and how did you decide which foods to add? a. What were the most important considerations involved in deciding what foods to add (eg, costs, nutritional guidelines, retailer feedback)? a. Of these, which were specific to the COVID-19 pandemic? b. What were the sticking points, if any? c. How long did it take to develop and adopt the expanded list? 4) How was information about the expanded food list communicated to local WIC agencies, retailers, and WIC clients? a. Other than these 3 groups, were there any other stakeholders that needed information about the list expansion? b. How easy or hard was it to get information out about the expanded list to all necessary stakeholders? c. Was information about the expanded food list highlighted through the WICShopper app in any way? 5) What feedback have you received about the expanded food list from clients, WIC staff, and retailers? Has the feedback been generally positive, generally negative, or mixed from these different stakeholder groups? a. Has any of the feedback led you think about additions or revisions that should be considered in the future? If so, what are some examples? 6) How and to what extent do you think the expansion of the food list has impacted WIC clients? a. To what extent do you think this may have been influenced by COVID-19 and changes in how people have been shopping for food or the availability of food items? b. How might the impact of the expanded food list be different in more typical, non-COVID, times? 7) Going forward, do you feel the food list should be expanded or refined further? a. If so, in what ways? b. If so, how important or urgent do you feel these changes are? 8) We know other states also expanded their food list in response to COVID. What have your heard from them, and have you reconsidered or refined your approach based on this?
Local WIC Staff
1) To begin, please share how WIC [certification and services such as nutrition education, breastfeeding support, and referrals] have changed at your sites since the pandemic began. a. [Program coordinators] How and where are appointments conducted? For example, are they happening in the office; curbside; or via phone, text, or chat platforms; GoToMeeting; or some other way? b. [Program coordinators] What methods are used to send and sign documents? c. [Certifiers and program coordinators] Is your agency receiving any bloodwork and height and weight information from participants? If so, how? d. How are educational materials provided to clients? 2) Think about the services your WIC site is currently offering remotely or differently in some way since the COVID pandemic. a. [Program coordinators] What new technology, equipment, or training was needed? b. [Program coordinators] To what extent have you been able to get this needed technology, equipment, or training? c. [All other local staff] What new technology or equipment have you have needed to use, and how do you feel about this new technology or equipment? d. [All other local staff] What training was provided? What has been most useful for you? 3) Think about how the changes to WIC services [certification, nutrition education, breastfeeding support] have been working for you and other WIC staff. a. What has been challenging and what has gone well? b. How have the changes impacted staff workload? c. How have the changes impacted the quality of WIC services, if at all? Are any tasks or services unable to happen or happening less consistently (eg, bloodwork, height/weight measurements, referrals to RDs, provision of educational materials)? d. What additional support is needed, if any? 4) Think about how these changes to WIC [certification, nutrition education, breastfeeding support] services have been working from the perspective of WIC clients. a. What do clients find frustrating or confusing and what do clients seem to like? b. What additional support do clients need? c. How have these changes to the WIC services impacted their convenience for clients, or the effectiveness of the services? d. Have you noticed any differences in how these changes have been experienced by specific groups of clients? For example, has the experience been different for tribal clients, clients that speak a language other than English, working parents, families of different sizes, or families who live in rural areas? 5) What changes or supports would you recommend to improve the quality, convenience, or impact of remote WIC [certification, nutrition education, breastfeeding support] services? 6) In general, would you like to see remote WIC [certification, nutrition education, breastfeeding support] services continue? If so, how important do you think this is? a. If you feel there should be a mix of remote and in person services be offered, what might that look like? 7) [Certifiers] As you know, the WIC program greatly expanded its list of eligible foods for purchase in April 2020. What feedback have you heard about this change from clients and retailers? a. To what extent do you think the change has impacted benefit redemption or client food security? Do you think this may have been influenced by COVID-19 and changes in how people have been shopping for food or the availability of food items? 8) We also plan to conduct interviews with WIC clients about changes made to WIC certification and services since COVID started. What questions you would suggest that we ask? 9) Please think about 1 or 2 successful client interactions you have had. Did they happen remotely? If not, could they have happened if services were offered remotely? Why or why not? 10) Those are all the questions we have today. Can you think of anything we did not talk about today that would be important to consider in relation to remote WIC services or the expanded food package?

Participant Interviews

The semi-structured interview guide for WIC participants was informed by the study aims, RE-AIM framework, and emerging themes from the staff focus groups. Questions were developed by the research team and reviewed by WA WIC. Participants were asked about their history with WIC, their most recent remote WIC appointments, their experiences with the expanded food list, their interest in continuing remote appointments, and how WIC could be improved in the future (Figure 3; available at www.jandonline.org). Interviews were conducted by the same researchers from March through April 2020 via Zoom and were recorded with interviewees’ permission. Participants were aware that researchers were affiliated with the local university and were working in partnership with the state WIC program. Interviews were conducted in English or Spanish (with a professional interpreter). Interviews with English-speaking WIC participants averaged approximately 20 minutes and interviews with Spanish-speaking WIC participants and interpreters averaged approximately 40 minutes. There were no repeat interviews with any participants. Interviewees were compensated differentially for this additional interview duration (English-speaking participants received $20 and Spanish-speaking participants received $40), but participants were not aware of the compensation differentials.

Figure 3

Semi-structured interview guide used with Washington Special Supplemental Nutrition Program for Woman, Infants, and Children (WA WIC) participants to determine their experiences and perceptions of programmatic changes during the COVID-19 pandemic. EBT = electronic benefit transfer.

WIC PARTICIPANT INTERVIEW QUESTIONS
Note: When we ask questions like, “When and where did you most recently get signed up for WIC” we mean you or your children.
Background:
1) When and where did you most recently get signed up for WIC?
2) [If applicable] Based on your answers to our online survey, it looks like you HAVE been on WIC before, is that correct? Was it from the same agency you are using now or a different one?
Enrolling in WIC:
3) When you signed up for WIC, was it in person, over the phone, or by video-chat?
4) Tell me little about the process of getting onto WIC from what you remember. What about it was easy or convenient? What was confusing, difficult, or inconvenient?
5) How did you learn about using your WIC card?
6) Do you use the WICShopper app? If yes, how did you learn to use it?
7) What is it like using the WICShopper app? What about using the WICShopper app was confusing, difficult, or inconvenient?
8) Before COVID, getting on WIC always happened in-person at the clinic. Since last March, many clinics have handled some or all parts of getting on WIC over the phone or by video-chat. What do you think about getting signed up for WIC by phone or video-chat?
Other WIC appointments:
Since you got signed up, have you had other communication with WIC staff? If yes:
9) Was the contact by phone, by text, by video-chat, in person, or a combination? a. What about this worked well? b. What about this was confusing, difficult, or inconvenient? Further probes:• How easy or hard was it to connect with WIC staff over the phone/video and hear WIC staff during appointments? • How comfortable did you feel talking with WIC staff over the phone/video during your appointments? [If they were on WIC before COVID:] Did you feel any more or less comfortable talking with WIC staff over the phone/video as compared to in-person appointments?] • [If they were on WIC before COVID:] Do you feel like you get more, less, or different information and support from WIC when appointments are over the phone or video? • [If they were on WIC before COVID:] Is there anything you miss about the in-person appointments? If so, what do you miss and why? • Have any topics been hard to talk through with WIC staff over the phone? 10) When appointments happen over the phone or video, do you make sure you’re at home at the scheduled time, or do you join the appointments wherever you happen to be? 11) Were you offered the option to have a video-chat appointment? If so, did you do it? Why or why not? a. What do you think about the idea of having appointments over video chat in the future? What might be helpful? What concerns do you have? 12) [If breastfeeding:] Were you offered breastfeeding support? If yes, how did that work for you? 13) [If primary language is non-English:] Was an interpreter used for any of your WIC appointments? If yes, how did that work for you? 14) Would you like WIC appointments to happen over the phone or by video after COVID? Why? 15) How will you feel if asked to come in-person for some or all WIC appointments after COVID?
Getting WIC foods:
16) During COVID, how easy or hard is it for you use your WIC card at the store? Why? a. Which foods are the most difficult to find or shop for? Why? b. Are there WIC foods you don’t buy every month? If so, which ones? Why is that? 17) [If they were on WIC before COVID:] WIC expanded its list of eligible foods in April 2020. Were you aware of this? If yes, what did you think about the expansion of the food list?
Other/Closing:
18) As you know, WIC provides moms and children with food benefits, nutrition education, breastfeeding support, and other types of referrals. What’s most helpful to you about WIC?
19) How concerned do you feel about the COVID pandemic? Would you say not at all, a little, moderately, or very concerned?
20) Due to the COVID pandemic, have you or other people in your household had difficulties with any of the following things? I am going to list off several items. Please tell me yes or no for each. a. Reduced wages, work hours, or lost jobi. If yes: Have you used paid annual leave, sick leave, unemployment benefits, or any other program to provide some wage replacement? b. Childcare or schools being closed, or having less access to them.i. If yes: How have you dealt with care for your children during this time? c. Difficulty with transportationi. If yes: What were the difficulties and how did you deal with them? d. Difficulty with getting or having enough of the food your family needsi. If yes: What were the difficulties and how did you deal with them? Has WIC helped you access food from other sources (beyond WIC)? e. Difficulty with paying the rent or mortgage f. Difficulty getting health care, including getting medications, or paying for medical costs 21) Do you have any children who got meals from childcare or school before the COVID pandemic? g. If yes: Have you continued to receive childcare or school or meals during the COVID pandemic? h. Have you received pandemic EBT during the COVID pandemic? By pandemic EBT we mean extra EBT funds because your kids used to get food at school.
22) Overall, do you have any suggestions for how WIC could work better for families in your community or families like yours?
23) Can you think of anything we didn’t talk about today that would be good for us to know?

Data Analysis

Programmatic data were analyzed in Microsoft Excel using descriptive statistics and visualized in Tableau to assess trends over time for total participation by women, infants, or children; percent increase and decrease in participation; percent of food benefits redeemed monthly; and appointment completion rates.

Focus Groups and Interviews

Focus groups and interviews were transcribed using the Zoom automated transcript function. Study team members reviewed and cleaned transcripts for accuracy, referring back to the audio recording as necessary. Transcripts were not reviewed by participants. The team used Dedoose software to apply deductive codes generated from the RE-AIM constructs, adapting the codebook to include emergent parent and child codes.

Four team members engaged in an iterative process of independently coding a small subset of the transcripts and then comparing code applications to refine code definitions. Coding inter-rater reliability between 2 pairs of coders was assessed in Dedoose; on reaching κ-statistics signifying “good” inter-rater reliability (ie, .79 or greater), team members coded independently. Data saturation, meaning no new themes emerged from the data, occurred before all transcripts were coded; however, the remaining transcripts were coded because the staff and participant samples intentionally captured a diverse set of experiences with WIC. After coding, team members summarized coded text.

Reflexivity, an evaluation of researcher’s experiences and identities that could influence the research is important in qualitative research. There were ways in which the study team’s experiences and identities differed from the study participants, which had the ability to influence study design, data collection, and interpretation. To account for these differences, the team debriefed regularly throughout the research process; wrote reflections after focus groups and interviews; consulted with co-researchers to ensure precise analysis; and discussed findings with WA WIC staff to cross-check interpretation, a process known as “member-checking.”

Results

Sample Characteristics

Characteristics collected for WIC staff and WIC participants are detailed in the Table . The majority of WIC participants interviewed for this study were enrolled with 1 or more children and had experience with WIC before the beginning of the COVID-19 pandemic. The majority of WIC staff who participated in interviews or focus groups were employed at the local level.

Table

Characteristics of participants and staff from the Washington WICa program who participated in interviews and focus groups on programmatic adaptations made in response to the COVID-19 pandemic

Category	n (%)
WIC participant characteristics (n = 40)
Race/ethnicitybc
American Indian or Alaska Native	3 (8)
Asian	3 (8)
Black/African American	3 (8)
Hispanic	14 (35)
Multiracial	3 (8)
White	21 (53)
Other	4 (10)
Missing	3 (8)
WIC enrollmentb
Self	17 (43)
1 or more infants	14 (35)
1 or more children	30 (75)
Urbanicityde
Urban	33 (83)
Rural	7 (17)
Geographyd
Eastern WA	13 (32)
Western WA	27 (68)
Experience with WIC
Received WIC services at some point before the COVID-19 pandemic (before March 2020)	35 (88)
Received WIC services for the first time during the COVID-19 pandemic	5 (12)
WIC staff characteristics (n = 52)
Agency jurisdiction/service area
State	10 (19)
Local	42 (81)
Urban	22 (52)
Urban and rural	13 (31)
Rural	5 (12)
Tribal	2 (4)
WIC staff role based on assigned focus groupf
State agency
Remote services	5 (10)
Expanded food list	5 (10)
Local agency
Program coordinators	14 (27)
Certifiers	11 (21)
Breastfeeding educators	5 (10)
Nutrition educators	6 (12)
Mixed staff roles	6 (12)

WIC = Special Supplemental Nutrition Program for Women, Infants and Children.

Categories are not mutually exclusive.

2020 Washington state race and ethnicity statistics: 78% were White, 13% were Hispanic, 10% were Asian, 5% were 2 or more races, 4% were Black, 2% were American Indian and Alaskan Native, and 1% were Native Hawaiian or other Pacific Islander.,

Based on WIC agency from which they received services.

Using the US Census Bureau and Office of Management and Budget definitions, SHARE-NW classified the Washington State counties into rural and nonrural and reported each county’s population; in 2013, 85% of Washington population lived in nonrural counties and 15% of Washington population lived in rural counties.

Many staff had multiple roles in WIC at their agency. Only 1 role is reflected here; categories are mutually exclusive.

Reach

Based on data reported by agencies and focus group respondents, all WIC participants who had an appointment or were certified during the COVID-19 pandemic experienced some aspect of remote services. Because the food list expansion was implemented statewide, all WIC participants also experienced the additions to the allowable food list. By May 2020, the WIC participant population reached with these changes included 26,891 women, 26,378 infants, and 72,010 children in Washington State; the population was predominantly White (70%), followed by multi-race (10%), Black or African American (8%), American Indian or Alaskan Native (5%), Asian (5%), and Native Hawaiian or Pacific Islander (2%); 42% of WIC participants identified as Hispanic.

Effectiveness

WIC Participation

As seen in Figure 4 , participation had been declining steadily since 2017. Participation began increasing just before the start of the COVID-19 pandemic and continued to increase after the programmatic changes were made to WIC in March 2020. There was a 5.9% increase in participation from December 2019 to December 2020, and participation increased by 2% from March 2020 (when the COVID-19 pandemic began) to December 2020.

Figure 4

Overall participation in Washington State Special Supplemental Program for Women, Infants, and Children (WA WIC) broken down by participant type (2017-2020). Participation data provided by WA WIC.

Between December 2019 and December 2020, participation increased for children by 11%, but decreased slightly for women (0.1% decrease) and infants (0.1% decrease). Participation growth also differed by race; from December 2019 to December 2020, participation increased among children identifying as Native Hawaiian or Pacific Islander (18% increase), Black or African American (17 % increase), multi-race (14 % increase), Hispanic (12% increase), White (11% increase), and Asian (9% increase), but participation decreased for children identifying as American Indian or Alaskan Native (6% decrease).

Appointment Completions

Local and state staff described increased appointment completion rates after implementing the programmatic changes, hypothesizing that the increase related to the convenience of remote services. As described by 1 WIC staff, “Our show rates are much higher because there isn’t the barrier of gathering all of your kids and your things and either driving or taking public transportation to get to WIC” (staff 02). Programmatic data from sample months before and during the implementation period corroborated this statement; nutrition education completion rates increased from 78% in June 2019 to 96% in June 2020, and the certification completion rate increased from 72% in June 2019 to 77% in June 2020.

Redemption of Food Benefits

Staff reported that the expansion of the allowable foods helped increase access to approved foods during COVID-19 pandemic–related food shortages. Participant interviewees echoed this statement. One participant shared that “at the beginning, it was hard to get milk, cheese, and cereal, but then WIC updated the list of available foods and now I am able to get them with no problem” (participant 30S).

Food benefit redemption data throughout 2020 (Figure 5 ) demonstrated that the average percent of all food benefits redeemed by WIC participants declined at the start of the COVID-19 pandemic from March to April, then increased almost back to pre-COVID-19 pandemic levels right after the food list was expanded. Redemption rates then dipped once more before returning to rates near those experienced in early 2020.

Figure 5

Average food package redemption rates (%) of Washington State Special Supplemental Nutrition Program for Women, Infants, and Children (WA WIC) participants by month in 2020. Food package redemption data provided by WA WIC

Adoption

State staff described 3 federal waivers that made remote services possible during the COVID-19 pandemic, listed in Figure 6 .30, 31, 32 These waivers allowed WIC services to continue while offices were closed to the public and reduced the administrative burden on clinics that had staff pulled away to COVID-19 pandemic response. In addition, WA WIC had transitioned to an electronic benefit transfer (EBT)–based system for benefit issuance throughout 2019 and had just started loading benefits remotely in the beginning of March 2020; WA WIC was able to continue remote benefit issuance throughout the transition to remote appointments. The other major programmatic change was a large-scale expansion of the approved WIC food list. The food list is updated annually, but typically 40 to 60 items are added; in response to COVID-19 pandemic–related food shortages and challenges finding WIC-allowed foods in the store, WA WIC staff approved approximately 600 new foods. WA WIC staff explained that they chose not to use the waivers related to food items so that added foods would not be revoked when the waivers expire.

Figure 6

Key federal waivers that the Washington State Special Supplemental Program for Women, Infants, and Children (WA WIC) adopted during the COVID-19 pandemic.

Adaptation	Explanation
Physical Presence Waiver30	Participants are allowed to certify and recertify for WA WIC services remotely (via phone or video appointment) and defer height, weight, and hemoglobin measurements.
Remote Benefit Issuance Waiver31	Staff are allowed to issue a participant’s benefits remotely.
Separation of Duties Waiver32	A single employee is allowed to evaluate a participant’s eligibility for all certification criteria, and issue food benefits.

Implementation

Remote Services Implementation

Phone appointments were mentioned by all staff as the most common way of reaching clients remotely. Although some local staff had attended state or agency-led training on using video conferencing, nearly all local staff said that video appointments had not yet been offered to WIC participants. Perceived challenges to offering video appointments included hesitancy from staff related to embarrassment interacting on camera, a lack of training, and limited access to video equipment. To help maintain contact with WIC participants, local staff, especially breastfeeding support staff, reported texting participants more frequently between appointments.

To supplement remote services, many local staff reported mailing handouts to participants either before some appointments. so participants could follow along with the content while on the call, or after appointments when participants requested additional information. Staff also discussed implementing pick-up services for breast pumps and educational materials. A small subset of staff mentioned putting infant scales outside of homes so parents could measure their child if growth was concerning.

Supports and Facilitators of the Implementation of Remote Services

Transitioning to remote services required numerous supports from federal, state, and local agency staff. A federal waiver to defer measurements combined with state-level policy to defer submitting proofs and signatures on documents improved the flexibility of the certifications. As a result, staff reported that appointment efficiency increased; one said, “when you’re doing everything over the phone things are just a bit quicker. You’re talking to them, but you can kind of write your notes simultaneously . . . so that saves a lot of time” (staff 28). State staff also initially provided biweekly webinars for agency staff to share information and assess needs related to virtual appointments; these webinars tapered to monthly by December 2020. At the local level, common requests from staff included cell phones, laptops with cameras, and attachable webcams for desktops.

Staff and Participant Satisfaction with Remote Services

Both staff and participants expressed high satisfaction with remote service delivery. Figure 7 summarizes a subset of the most common advantages and disadvantages of remote services from staff and participant perspectives and presents a selection of illustrative quotes. The advantages of remote services were mentioned more often and outweighed the disadvantages presented.

Figure 7

Perceived advantages and disadvantages of conducting appointments remotely during the COVID-19 pandemic as reported by Washington State Special Supplemental Program for Women, Infants, and Children (WA WIC) staff focus group participants and WA WIC interview participants. This figure includes a subset of the most common advantages and disadvantages of remote services noted by both staff and participants; it is not an exhaustive list. Key quotes have been selected to represent the theme presented. Participants with S after their ID number were interviewed in Spanish, participants with an E after their ID number were interviewed in English.

Theme	Quote
WA WIC staff focus groups
Advantages of remote services
Increased convenience for working parents, large families, those with transportation barriers	“I think the majority of our clients like it, because it’s quicker and they don’t have to come in.”—staff 28
Decreased barriers to participation such as:• Having to take time off work for appointments • Having to bring multiple children to office for one child’s appointment • Having to secure access to reliable transportation • Having to travel a long way to reach the WIC office in remote/rural areas	“All of the clients that I’ve talked to really appreciate that they don’t have to get in the car, load all the kids up, and go to the clinic.”—staff 33
Increased ease of sharing resources virtually with participants	“They find [links] very helpful because after they had the education with the peer counselors or the breastfeeding staff the mom is able to go back and review that video again for anything that she had missed.”—staff 02
Increased amount of communication between staff and participants (ie, between appointments and for questions)	“People used to save up their little questions for the end of an appointment and go ‘Oh, there was something else I wanted to ask’ that they couldn’t remember and now they’re texting me more than they used to with just little questions intermittently.”—staff 29
Increased richness in conversations over the phone	“I think in part of the certification they have richer discussions. Participants are willing to share a little bit more over the phone versus face to face. —staff 02
Increased ability to adapt services to address participants needs in the moment	“I was able to save that breastfeeding relationship by being able to meet that mom in that moment when she needed help right then with feeding. So that was really great for remote services.”—staff 29
Disadvantages of remote services
Difficulty obtaining consistent height, weight, and hemoglobin measurements for infants and children	“It just feels like you’re flying without wings on not having accurate measurements and timely measurements.”—staff 25
Missing in-person relationships and interactions with participants	“Seeing clients in person you get to see how they’re feeling in their expressions when they’re talking to us and you don’t pick that up remotely when you’re just talking on the phone or even in a text and so we tend to miss out on a lot of information that we could pick up if we were with them in person.”—staff 29
Increased difficulty of some appointment types (eg, new participants and participants using interpreters)	“Those new certifications are harder. I find I take probably 15 to 20 minutes more than I used to.”—staff 40
Difficulty engaging participants without handouts and materials to share and describe in-person	“Not be able to have like visuals that I usually have when I’m explaining how to use the [WIC EBT] card . . . or while they’re in my office showing them how to download the app on their phone . . . not being able to help them with that I think some people struggle.”—staff 26
Noticing at times participants seem distracted when taking appointments over the phone	“Being on the phone it’s kind of hard to try to get like the client’s needs because they’re just kind of wanting to either just hurry up with the appointment because their kids are loud or they can’t hear us.”—staff 25
Increased difficulty scheduling and dividing the caseload fairly for select agencies	“Sometimes a certifier will get four of those five appointments at the same time versus, you know, somebody else who will get a nice spread-out schedule. When you have as many as 100-250 200 clients a day, you can’t go through and just individually piece it out.”—staff 11
WA WIC interview participants
Advantages of remote services
Not having to secure childcare or bring children to appointments	“It was more convenient for me to do it over the phone because, with my kids, that meant I didn’t have to bring them or find a babysitter for them. I was able to just be at home and have all the information ready for it.—participant 49E
Saving on travel time and cost of transportation	“It’s just convenient. You know, I don’t have to drive all the way to the actual office itself.”—participant 107E
Not having to take time off from work	“I can have my with appointments on calls from anywhere . . . I’m able to you know, be at work.”—participant 93E
Accomplishing other tasks during appointments	“I can have my with appointments on calls from anywhere . . . I’m able to, you know, be at work and take the call or you know doing whatever it is and I’m not having to get up to go somewhere and all of that so it’s pretty great.”—participant 93E
Feeling safer during the pandemic	“It’s actually better, it’s safer for people to be at home and especially because going to the WIC office, the office is really small, so they wouldn’t be able to keep the social distancing.”—participant 30S
Appointments seeming more focused and personalized	“Most of the time, it’s the same lady that calls me. So I mean . . . it kind of gets a little personal because we kind of get to know each other over the phone.”—participant 87E
Disadvantages of remote services
Missing information on height, weight, and iron for infants/children	“Before, when we went in in person, we would know how the baby what was thriving and his weight and right now on the phone we wouldn’t be able to know.”—participant 25S
Missing interactions between staff, other parents, and children	“She misses the ‘Hello, how are you? How’s your daughter?’ Those interactions that she enjoyed while being in person.”—interpreter for participant 9S
Feeling some appointments seemed rushed or brief	“On the phone, it is kind of just quick like, ‘Well, we’re going to add this, and this is going to happen like this, and we’ll see you talk to you in like 3 months or something.’ So, I don’t know if we miss anything, but it’s just really different.”—participant 70E
Feeling that some appointments might not allow enough space and time to observe and interact with children, especially for new parents	“I feel like they give more information when the child is in the room, and then they can see, ‘Oh your child can do this. Your child can do that.’ So, they can provide more information about specifically my child.”—participant 36E

Both local and state staff appreciated remote services as a way to reduce longstanding participation barriers. Phone calls worked well for most appointment types, according to most local staff. The appointment types that were mentioned as more difficult remotely included enrollment appointments (especially for first-time WIC participants) and some breastfeeding appointments addressing latch and positioning support. For example, one staff member shared that “it’s been very challenging offering these breastfeeding support services remotely. It’s hard to . . . convey the different positions or movements just, you know, over the phone” (staff 29). Although enrollments and breastfeeding support appointments were brought up by some staff as challenging, these opinions were not unanimous. For example, one breastfeeding educator said “I feel like even for breastfeeding they’re more relaxed when they are talking, because . . . we have the luxury to choose any time at their convenience and they can call me” (staff 39); in this way, remote services helped the participant feel comfortable and engage with breastfeeding staff when they needed it. The most reported element of remote services that detracted from staff satisfaction was the inability to consistently obtain height, weight, and hemoglobin measurements from participants. However, these challenges were mentioned less often than the success of increasing convenience for participants with remote services.

Participants viewed phone appointments as easy to complete, convenient, and sufficient for both sharing and receiving information. Participants reported they could connect, hear, and complete the appointments over the phone and felt as, or more, comfortable receiving services on the phone compared with in-person. Approximately one-half of the participants interviewed indicated they occasionally took appointments away from home, including at work and while running errands. One participant noted that phone appointments “just made it really easy. It was convenient that as I’m doing my normal everyday things that I can still do my appointment” (participant 36E). None of the participants interviewed had conducted a video appointment and thus did not discuss satisfaction regarding video appointments.

Satisfaction with the Expanded Food List

When asked about their experience with the expanded food list, approximately one-half of participants were aware of the additional foods. Those participants were happy with the expansion and especially appreciated the increased variety of kid-friendly foods like string cheese, yogurt, cereal, milk, and juice. One participant said “Yes, I like them, because we have more and more of an option of how to get cheese and yogurt because that’s what [my son] likes” (participant 25S).

The WICShopper app was mentioned by participants as helpful when planning for and actively shopping, especially alongside the expanded food list. The scanning feature made it easier for participants to confirm allowable foods. Although the WICShopper app was useful for a majority of the participants interviewed, some still noted difficulties around determining which foods were added to the food list and having items deemed not allowable at the register.

Maintenance

Interest in Maintaining Remote Services

Both staff and participants expressed hope that remote services would continue beyond the COVID-19 pandemic. Almost all state and local staff advocated for a hybrid model, including both remote and in-person appointments. They recognized the increased convenience of remote appointments, and also noted that an in-person option could be useful for certain services and for parents needing to share sensitive information, including disclosures of domestic violence. One staff member shared that “moms sometimes like to come and share what’s happening in their homes and . . . the kind of abuse they were facing or things like that . . . so a hybrid fashion, I think, is a better idea” (staff 12). Although an in-person option is desired, many staff anticipated a decline in caseload if reverting back to all in-person appointments. WIC participants generally said they would be fine with returning to in-person services, although one-third of interviewees said they would want to know returning was safe in terms of COVID-19 pandemic protocols and another one-third said they would return if it was required.

Nearly all participants interviewed hoped that remote services would continue after the COVID-19 pandemic. The most common reason given was the improved convenience of the phone appointments. Most participants supported an option for in-person appointments. For example, one participant suggested that “as far as reloading the card, then yeah, phones fine. But if it has anything to do with checkups or with growth information, then I would like those to be in-person” (participant 25S). Nearly one-quarter of WIC participants proposed that in-person appointments be provided for, or limited to, appointments with specific purposes like certifying, breastfeeding support, and/or discussing sensitive topics. Four participants—2 English-speaking participants and 2 Spanish-speaking participants—said that they would prefer all in-person services. Participants indicated that in-person appointments would allow more opportunities for socializing and observation of breastfeeding skills or child health. On this latter point, a participant said “To be honest, after COVID, [I] would like to go to the clinic because then they could assess the children. They could actually physically see the children and be able to evaluate them” (participant 57S).

Staff Suggestions for Improving Remote Services

The staff proposed many ideas for creating a hybrid service delivery model, including drop-in days for height and weight measurements and conducting the rest of the certification appointment remotely, and/or conducting biannual certifications in-person and all other appointments remotely. Staff also had ideas for capturing measurements without requiring physical presence, including improved data sharing among health providers and programs that capture measurements and technology for at-home measuring. Staff expressed strongly that consistent and accurate measurements are essential to assessing nutritional risk and, further, that WIC is unique in its ability to collect and use this information, especially among families that do not have insurance.

Supports Needed to Maintain Remote Services

Staff described federal- and state-level supports that would be necessary to continue providing remote services. All staff want to see flexibility in federal physical presence regulations. State staff also recognized the importance of further training on video conferencing, describing a significant learning curve for both staff and participants. Approximately one-quarter of staff expressed interest in conducting more video appointments, and participants had mixed and speculative views on video appointments because none had yet completed a video appointment. From the participant perspective, perceived advantages included seeing demonstrations, having staff interact with children, and feeling more comfortable seeing the staff member. The most common perceived disadvantage of video calls described by participants was that they would not be able to take calls “hands free,” which was an important convenience of phone appointments. Staff also suggested developing video orientations for using the WICShopper app and WIC EBT card translated into numerous languages to help supplement the education they give to new participants during their first appointment.

Interest in and Improvements Recommended for a Maintained Expanded Food List

Participants made clear that they would like the list of allowable food to remain expanded, with even more foods added. Participants said they would appreciate additional approved food brands, more organic options, and more flexibility in allowable food package sizes. Some WA WIC participants said that although the number of approved brands of certain foods increased, the allowable sizes of food items remained a constraint because food packaging sizes in stores differed. Participants also suggested improvements to the WICShopper app, including improved search features, increased scanning reliability, and additional meal planning features. Both staff and participants mentioned that participants should be able to order WIC groceries online and pick up curbside.

Discussion

Public health nutrition experts have called for evaluation of WIC service changes implemented in response to the COVID-19 pandemic to strengthen WIC and inform the upcoming Child Nutrition Reauthorization process. This study was designed to generate evidence-based suggestions for implementing, maintaining, and improving the changes adopted by WA WIC for future practice. Overall, these results illustrate that appointment show rates improved after the programmatic changes were implemented and that WIC participants and staff were highly satisfied. Staff and participants communicated strongly that they want to see both changes maintained in some form. The impact of these changes on WIC participation trends and food benefit redemption is less clear, given the COVID-19 pandemic context influencing WIC eligibility and food shopping experiences; increased unemployment, increased food insecurity, and decreased access to other food-related safety net programs, make it difficult to determine the extent to which increases in participation and food benefit redemption are attributable to WIC programmatic changes as opposed to increased need. However, the changes demonstrate promise for optimizing WIC services beyond the COVID-19 pandemic.

This study aligns with other research on WIC participants’ experiences with remote services during the COVID-19 pandemic and delves deeper into perceived advantages of remote appointments. Participants in Tennessee and now Washington State have indicated that phone appointments have been more convenient than in-person appointments. Similar to participants in California, WA WIC participants felt comfortable with phone appointments and sharing documentation remotely. Uniquely, participants in this study highlighted specific advantages of the remote appointments that address longstanding barriers to WIC; the most notable benefit was not having to secure childcare for appointments or bring multiple children to appointments. Past research has demonstrated that bringing children to appointments is one of the most common barriers to recertifying. Other advantages mentioned by participants in this study that address barriers to using WIC services include saving on travel time and costs, taking appointments on breaks from work, and feeling like appointments were more personalized on the phone. Similar to findings from other states, , WA WIC participants were highly satisfied with remote services and explicitly suggested their continuation.36, 37, 38

Findings from this research also showcase satisfaction with the expanded food options during the COVID-19 pandemic, and suggest that the expanded food list likely offset a further decline in food benefit redemption rate due to COVID-19 pandemic–related food shortages. Food benefit redemption increased from April to May 2020, after the food list was expanded; while it did decrease again from May to June and July, this could have been related to the increase in Supplemental Nutrition Assistance Program benefits and the COVID-19 pandemic EBT disbursements that began in late June and early July in Washington, which likely impacted the purchasing patterns of many WIC participants. This study also provides unique insight into participants’ opinions on expanded foods that were most appreciated during the COVID-19 pandemic, including string cheese, yogurt, milk, cereal, and juice. WA WIC expanded these categories without using waivers, so these additional items would not be revoked when waivers expire. To help determine which new items in the store were WIC-allowed, both California and WA WIC participants found the WICShopper app particularly helpful. Despite the expanded food list and WICShopper app, participants from the current study noted that package size constraints were magnified during the COVID-19 pandemic. Package size availability has been a longstanding challenge reported by WIC participants, and this current study indicated it has continued to impede WIC benefit redemption even within the context of expanded food options. Current study findings add further support for policy change regarding package sizes of WIC approved foods.

This is the first study that presents the WIC staff perspective on how services changed during the COVID-19 pandemic and related suggestions for future practice. Staff perspectives were similar to WIC participant perspectives when describing the ease and convenience of remote services, aspects of remote services that decrease participation barriers, and the helpfulness of the WICShopper app. The main drawbacks to remote services outlined by staff also aligned with the participants’ perspectives; they wanted a way to capture consistent measurements remotely and they missed interacting with WIC participants in the clinic. Although there were similarities between staff and participant perspectives, they also diverged in several areas. First, some staff reported participants occasionally seemed distracted while taking phone appointments away from home. Participants did discuss taking appointments away from home, but within the context of convenience, highlighting the ability to multitask or maximize breaks from work as a benefit of remote services. Second, staff anticipated attrition once in-person services were re-established. Participants did report wanting to continue receiving WIC services remotely after the COVID-19 pandemic, but none said they would leave the program if in-person services were required. Staff and participants had similar suggestions for programmatic changes in the future, including transitioning to a hybrid model and including online ordering and pick-up options. Staff perspectives should be incorporated into future research, when possible, as it is valuable to know where WIC staff and participants’ priorities align in order to make recommendations for changes.

Study findings add to the evidence supporting continued access to remote WIC services in addition to in-person services. , , However, maintaining an option for remote certifications beyond the COVID-19 pandemic would require federal policy changes. For participants to be able to choose to certify and recertify remotely, either by video or phone, the federal physical presence requirement would need to be amended to allow either hybrid or full-remote options in addition to in-person. Some of the rationale for the physical presence requirement are to observe WIC participants, identify health concerns, personalize service delivery, and protect against fraud. Video appointments could be a substitute for physical presence because they allow for participant observation and providers can successfully identify health concerns, as seen with the increasing use of videoconferencing in telemedicine. Furthermore, some participants in this current study felt that remote appointments were more personalized compared with in-person appointments and would like this option to continue in addition to in-person services.

To facilitate the continued success of remote appointments, additional means of collecting height, weight, and hemoglobin measurements should be piloted and evaluated; one option to research is drop-in hours for collecting measurements. Staff also suggested implementing data-sharing protocols between WIC and health care providers to minimize the duplication of services among participants whose measurements are being captured elsewhere. In addition, training and technological resources to maximize remote service quality in the future should be strengthened. , , WA WIC staff advocated for additional video trainings for both staff and participants to feel comfortable providing and receiving services via video, as well as prerecorded video orientations in multiple languages about shopping with the WIC EBT card and WICShopper app. To improve the WIC shopping experience and food security, state programs should expand their approved food lists as much as possible within the current federal regulations. However, federal policy changes will be needed to address food package size discrepancies. Moreover, any change to WIC service delivery should be tracked and evaluated for both successes and unintended consequences.

There are multiple strengths of the design, methods, and analysis of this study. Key strengths include the mixed-methods design; participation of a diverse participant sample, including staff perspectives; the iterative approach to coding qualitative data; and the member-checking process with WA WIC staff. Integration of programmatic data with qualitative data allowed for contextualization and explanation of quantitative trends; use of both quantitative and qualitative data within the RE-AIM framework has been recommended and both were employed in this study for a rigorous exploration of the changes. Furthermore, having multiple coders code a subset of the transcripts, discuss discrepancies, and collaborate on themes enhanced the data triangulation process , and confirmability of these results. The participant sample also captured a diversity of perspectives about the programmatic changes. Finally, in line with principles of rigor in qualitative research, the research team shared a preliminary synthesis of the results with WA WIC staff participants and discussed whether the findings resonated with their experiences.

There are also limitations to this study. First, the study focused on one state’s WIC program, limiting the national generalization of the results. Second, interviews were only conducted in English and Spanish due to cost constraints; adding more languages could have increased the WIC participant sample diversity. Third, some trends, such as increases in participation, may have already been under way before COVID-19 pandemic shutdown orders; the team did not collect information that would elucidate whether these trends were due to growing awareness of the pandemic in late 2019/early 2020 or to simultaneous programmatic changes, such as changes in electronic enrollment systems. Fourth, the team engaged in member-checking with WA WIC staff; however, there was not time to conduct a member-checking process with the participant sample. Fifth, demographic data on race and ethnicity, gender, or age of WIC staff participants were not collected and WIC staff from diverse racial and ethnic groups were not purposively sampled. The interpretation of study findings may have been enhanced in the context of WIC staff participant race and ethnicity, gender, or age. Sixth, and finally, the WIC participant sample was recruited through the WICShopper app, which may have generated a sample that would be more comfortable with the technology and experience of remote service delivery; however, nearly all WIC participants in Washington State use the WICShopper app and all have experienced remote services, so the potential for a biased sample is low.

Conclusions

This study explored the reach, effectiveness, adoption, implementation, and maintenance of remote services and the expanded food list executed by the WA WIC program in response to the COVID-19 pandemic. After these changes were implemented, participation, appointment show rates, and food redemption increased. Both staff and participants were highly satisfied with remote services and envision a combination of remote and in-person services offered in the future. Participants appreciated the increased variety of kid-friendly food options. Some of the suggested WIC changes, especially the continuation of remote certifications, would require federal policy change, and others, including increased training for video appointments and additions to the app and food list, would not.