Organizational contexts, implementation process, and capacity outcomes of multicultural, multilingual Home-Based Programs in public initiatives: A Mixed-Methods study.

AIMS: The study aim was to examine the impact of a home-based programme intervention on organizational contexts, implementation processes and organizational capacity outcomes from multicultural, multilingual participants working at community-based organizations.
DESIGN: This was a sequential exploratory, mixed-methods longitudinal study using community-based participatory research principles. SAMPLE: Twenty participants from nine multicultural, multilingual community-based organizations were in this public health initiative's intervention to develop community-designed, home-based programmes.
METHODS: Capacity building providers delivered the intervention selected by the funders. Workshop outcomes were descriptively measured in April/May 2019. In April/May and November 2019, participants completed surveys about organizational contexts, implementation processes and organizational capacity outcomes, which were analysed with t-tests using the organization as the unit of analysis. Qualitative data were analysed using content analysis.
RESULTS: Seven programmes were new and two were modified. As workshop outcomes, 59% of participants reported increased overall implementation knowledge and 74% reported capacity building providers as the most helpful resource. After 6 to 7 months, no statistically significant changes were noted in organizational contexts, implementation processes or organizational capacity outcomes. Participants benefited from capacity building because they had programmes developed, formed partnerships with capacity building providers, gained implementation knowledge, and engaged in networking.
CONCLUSION: Participants reported excellent individual and organizational strengths. Many Initiative factors contributed to no statistical changes. Namely, there was no opportunity for baseline data; limited community-based organization engagement in the intervention model selection, timeline and processes; the Initiative's timeline did not fit participants' timeline; insufficient time to develop culturally and linguistically appropriate programmes; late literature review abstracts; lack of adequate, planful and paid capacity building time; and a contract requirement to have the programme due when it was not implementable. These Initiative design factors, as reported by participants, limited the Initiative's home-based programme development. IMPACT: This study highlights the strengths of participants, community-based organizations and capacity building providers. Model selection, timeline and budget were identified as key factors for equitable implementation in multicultural, multilingual organizations.

INTRODUCTION

Young children born to multicultural, multilingual families in the United States can reach their developmental potentials (Britto et al., 2017). Implementing evidence‐based home‐based programmes in local community‐based organizations (CBOs) can promote pregnancy and early childhood health equity for those families (Hiratsuka et al., 2018). Effective implementation of home‐based programmes in local CBOs requires building sufficient organizational capacity. Organizational capacity building includes resources, structures and adequate workforces to plan, intervene and evaluate the evidence‐based programmes in CBOs (Brownson et al., 2018). Capacity building is an essential strategy for international health promotion as well (Rissel, 2005).

Previous organizational capacity building studies examined changes in individual knowledge, skills, leadership, confidence, practices and policies, behaviours, application, organizational support and perceptions of system level capacity building (DeCorby‐Watson et al., 2018). Research gaps included limited inquiry about organizational outcomes, baseline conditions and factors for successful interventions (DeCorby‐Watson et al., 2018). Understanding the contexts, processes and outcomes of capacity building among multilingual and multicultural CBOs in the current study can inform similar public health initiatives working with similar communities.

Background

Home‐based programme interventions for early childhood development

Meta‐analysis indicated that home visiting by supportive home visitors was effective in four areas: (1) preventing and decreasing low birth weight and rapid repeat birth (Liu et al., 2019), (2) improving positive infant‐care behaviours (Hans et al., 2018), (3) promoting the quality of the home environment and parenting (Kendrick et al., 2000), and (4) reducing all‐cause maternal mortality and infant preventable‐cause mortality (Olds et al., 2014). When home visiting programmes were used among tribal (Hiratsuka et al., 2018) and Latino communities (Beasley et al., 2014), cultural adaptations were required to meet community needs. In both of the previous studies, cultural sensitivity (Resnicow et al., 2000), which considers surface structure (e.g., language) and deep structure (e.g., sociopolitical influence) was used.

Implementation science framework for the Initiative's home‐based programme (HBP) intervention

Implementation science is defined as studying the systematic application of evidence‐based interventions into routine practice to enhance health services quality and effectiveness (Eccles & Mittman, 2006). The Initiative's home‐based programme (HBP) intervention in this study used the National Implementation Research Network (NIRN) model (Fixsen et al., 2018). The NIRN model includes the following components: (1) Well‐Defined Programme, (2) Effective Implementation and (3) Supportive Environments to create improved outcomes. For the Initiative's HBP intervention, capacity building providers (CBPs) supported equity practices in their work scope. Equity practices are defined as creating opportunities for cultural or social groups to minimize disparities using fair and just processes. The focus of equity practices was to reduce inequities during the Initiative's HBP intervention by welcoming multilingualism and multiculturalism. With equity practices, the hope was to ensure equitable implementation processes and capacity development for the local CBOs.

Organizational contexts

Organizational contexts, not treatment effectiveness, determine whether or how quickly an evidence‐based programme is implemented (Ferlie et al., 2005). This article addresses organizational strengths and organizational conditions during the Initiative's HBP intervention. When organizations, rather than individuals, are committed and confident (Weiner et al., 2009), have resources (Brownson et al., 2018; Lala et al., 2016), feel comfortable and resilient (Witmer & Mellinger, 2016), they successfully adopt changes. We define the preceding conditions as organizational strengths. Organizational conditions are defined as facilitators of organizational change including the following: (1) workload (Lala et al., 2016), (2) staff support (Brownson et al., 2018; Lala et al., 2016), (3) communication in organizations (Agarwal & Garg, 2012), (4) communication with funders (Rajhans, 2018), (5) partnerships with families and communities (Bryson et al., 2017; Witmer & Mellinger, 2016), (6) satisfaction with organizational systems (Brownson et al., 2018), (7) executive leadership support (Bryson et al., 2017; Mosson et al., 2019; Witmer & Mellinger, 2016) and (8) sustainability planning (Brock et al., 2019).

Implementation processes

Implementation processes affect implementation (Miake‐Lye et al., 2020). This article describes institutional procedural discrimination, capacity building guiding equity principles and ease of information as implementation processes. Institutional procedural discrimination is defined as the institutional agreements about the types of decisions, timing of discussions, conditions and deadlines to privilege one group over another in institutional processes (Lim et al., 2022). The concept of institutional procedural discrimination is derived from procedural justice theory that posits one's satisfaction with legal or clinical interactions is primarily influenced by the quality of the procedures rather than the outcome of the experience (Kopelovich et al., 2013; Kunard & Moe, 2015). Interactions that are perceived as procedurally just involve respect and dignity, involvement in the decision‐making process, trust in process fairness and absence of coercion (Kopelovich et al., 2013; Kunard & Moe, 2015). About 50% to 75% of Blacks, Hispanics and Asians reported discrimination and non‐White participants reported more discrimination than White participants (Lee et al., 2019). Institutional procedural discrimination, as well as racism, are noted as barriers to cultural competency and cultural safety (Berg et al., 2019).

Capacity Building Guiding Equity Principles are defined as the core values for guiding actions developed by funding staff and CBPs. Capacity building guiding equity principles include racial equity, cultural responsiveness, intersectionality, transformational, trauma informed, sustainability, continuous learning, relationship based, information informed and contextual responsiveness. Capacity building guiding equity principles are commonly used when working with CBOs, especially in community‐based participatory research (Alexander et al., 2015; Brakman, 2020). Capacity building guiding equity principles helped to identify enabling factors for successful public–private partnerships in seven countries (Pérez‐Escamilla, 2018).

Ease of information is defined as the degree to which information is easily understood and communicated in oral and written language. It is important to be able to read the workshop materials, listen and speak at workshops, and write the deliverables/assignments. A previous study found that the NIRN model provided a clear implementation framework, however it was highly detailed and time intensive to learn the language and review the different worksheets (Brémault‐Phillips et al., 2018). Specifically, the language used was developed in educational systems. The NIRN model did not fit well for healthcare systems so researchers suggested developing project‐specific NIRN tools for future studies (Brémault‐Phillips et al., 2018).

Organizational capacity

Organizational capacity outcomes include the: (1) 15 capacity building areas, (2) Culturally Responsive NIRN Practice Profile, (3) theory of change, (4) use of the literature review abstracts and (5) organizations better off from capacity building. The 15 capacity building areas include recruiting, hiring, training and measuring performance of the staff (Fixsen et al., 2018). The areas also included completing deliverables, developing/adapting programmes (Hiratsuka et al., 2018) and engaging families to inform evidence‐based programme development/adaptation (Bryson et al., 2017). Progress in the Culturally Responsive NIRN Practice Profile is defined as the progress in developing a home‐based programme. CBPs added culturally responsive content to the NIRN Practice Profile, which was a part of the Well‐Defined programme deliverables; thus, culturally responsive is placed in front of NIRN Practice Profile to indicate this change.

Additionally, we studied the theory of change, which is defined as the short‐ term and intermediate‐term goals developed by the funding staff and CBPs, with coordination support from the research team. In the NIRN model, the theory of change maps how the intervention elements and activities create improved short‐, intermediate‐ and long‐term outcomes for populations (Bertram et al., 2013). It helps plan activities, relevant outcomes, intended goals and pathways as well as evaluate engagement activities (Gooding et al., 2018). Challenges in developing the theory of change included considering the backgrounds of populations such as age, gender, race, ethnicity, socioeconomic and cultural factors (Bertram et al., 2013). Other concerns were identifying feasible and agreed on goals, and evaluation plan timelines (Gooding et al., 2018). The theory of change is widely used in public health research and practice; however, descriptions about its use is limited (Breuer et al., 2016). We fill a research gap by reporting about the use of the theory of change.

Literature review abstract use is defined as participants' perceptions of the use of the literature review abstracts to develop their evidence‐based home‐based programmes. Finally, organizations better off from capacity building is defined as participants' perceptions on how their organizations were better off from capacity building.

THE STUDY

Aims

The study aim was to examine the impact of the Initiative's HBP intervention on organizational contexts, implementation processes and organizational capacity outcomes from the perspectives of multicultural, multilingual participants working at CBOs. The primary research question was: How does the Initiative's HBP intervention affect CBOs': (1) organizational context, (2) implementation processes and (3) organizational capacity outcomes? We also report workshop outcomes about implementation knowledge, overall workshop satisfaction and overall resource use. Table 1 summarizes the research design.

TABLE 1

Data collection timeline, implementation interventions and outcomes

November 2018 (T1)	April/May 2019 (T2)	November 2019 (T3)
Initiative's Home‐Based Programme (HBP) Intervention 6 monthly workshops, Homework including deliverables, Individual consultations, Literature review abstract use
No real time baseline data were collected before the intervention start date in November 2018 because participants in the Initiative's programme were not finalized until February 2019	Organizational context outcomes ‐ Organizational strengths ‐ Organizational conditions	Organizational context outcomes ‐ Organizational strengths ‐ Organizational conditions
	Implementation processes outcomes ‐ Institutional procedural discrimination ‐ Capacity building guiding equity principles ‐ Ease of information	Implementation processes outcomes ‐ Institutional procedural discrimination ‐ Capacity building guiding equity principles ‐ Ease of information
	Organizational capacity outcomes ‐ 15 capacity building areas ‐ Culturally Responsive NIRN Practice Profile ‐ Theory of change a	Organizational capacity outcomes ‐ 15 capacity building areas ‐ Culturally Responsive NIRN Practice Profile ‐ Theory of change ‐ Use of literature review abstracts ‐ Organizations better off from capacity building
	Workshop outcomes a ‐ Overall Implementation knowledge ‐ Overall workshop satisfaction ‐ Overall resource use	N/A

Retrospective theory of change data and retrospective workshop outcomes were collected in T2.

Workshop outcomes

Design

This study used a sequential exploratory, mixed‐methods (Kajamaa et al., 2020) longitudinal study design, with community‐based participatory research principles (Vaughn et al., 2017). Researchers used four phases: (1) collecting and analysing qualitative data, (2) developing instruments that included open and closed ended survey questions using the qualitative data, (3) collecting and analysing both qualitative and quantitative survey data and (4) integrating/linking both qualitative and quantitative data.

Initiative's home‐based programme (HBP) intervention

The Initiative's HBP intervention was planned by the public health funders and delivered by CBPs. The goal of the Initiative's HBP intervention was to develop a home‐based programme by providing public health and capacity building support (Best Starts for Kids, 2018). Home‐based programmes were defined as relationship‐based supports for pregnant or parenting families of children 0–5 to improve maternal‐child health outcomes. The Initiative funded a programme manager. The Initiative also funded four different local CBP groups and one out‐of‐state NIRN model group to provide technical support. The local CBPs were: (1) a well‐defined programme CBP team, (2) a data support CBP team, (3) an organizational capacity building support CBP team and (4) a supportive environment CBP team. These teams were based on the NIRN model (Fixsen et al., 2018) to build organizational capacities related to Well‐Defined Programmes, Effective Implementation and Supportive Environments. Out‐of‐state NIRN model experts were contracted to teach the NIRN model to the local well‐defined programme CBP team. This CBP team, experts in working with diverse CBOs, thereafter would teach the NIRN model to the 10 CBO grantees. The public health funders contracted external researchers to examine the Initiative's HBP intervention.

The Initiative's HBP intervention during Phase I included the NIRN model content delivered through: (1) six monthly workshops, (2) homework and deliverables, (3) individual consultations and (4) literature review abstracts. First, after a launch meeting, participants attended six consecutive monthly in person workshops that lasted about 5 hours, including lunch. Workshops were held at a community centre from December of 2018 to May of 2019 to teach the NIRN model and other concepts. The well‐defined programme CBP team taught participants to: (1) create a racial equity theory of change, (2) recognize the concept of implementation science and create a culturally responsive NIRN practice profile, (3) engage community stakeholders and (4) incorporate equity concepts in the home‐based programmes. Public health staff led discussions about how to: (1) develop an implementation plan and (2) identify components of a 2‐year budget. The CBPs had extensive educational and work experiences with CBOs, serving as respected local expert consultants. They were trained to deliver research‐based education and services by working with diverse participants. As possible, they applied a racial equity and social justice lens in their work scope. The CBPs delivered workshop topics in English and presented topics using lay language to ensure multicultural and multilingual participants easily understood content.

Second, participants completed homework after monthly workshops that included the following four deliverables: (1) a racial equity theory of change (due in 28‐2‐2019), (2) a culturally responsive NIRN practice profile (due in 30‐4‐2019), (3) an implementation plan (original due date 31‐5‐2019, extended due date 15‐6‐2019) and (4) a 2‐year budget (original due date 31‐5‐2019; extended date 15‐6‐2019). Participants also submitted an evaluation plan (originally due 31‐5‐2019; some had an extension until July 2019). At workshops, participants received packets to support their homework (total of six packets from the six workshops). Some example articles are included in the Online Only Table 1. Third, CBPs spent an average of 4 hours per month providing in person consultations per CBO team, supporting programme development and answering questions about deliverable requirements. Fourth, in June of 2019, participants received literature review abstracts about home‐based programmes prepared by a maternal and child consultant contracted by the funder.

Phase II focused on initial home‐based programme implementation. Phase II also included developing capacities on data systems, organizational capacity and supportive environments. The data and organizational capacity building CBP teams started working with CBO teams at various times in the summer of 2019, depending on readiness to start Phase II. To ensure intervention integrity, funding staff and CBPs met biweekly to plan workshops, and one researcher also met with them to facilitate developing workshop outcome questions. The same researcher attended all workshops, 90% of the biweekly planning meetings, and wrote minutes. The research team thoughtfully studied the Initiative's HBP intervention, implementation processes and participants' experiences. See the Online Only Table 2 for the Template for the Intervention Description and Replication (TDIer) Checklist. Refer to the Online Only Table 3 for the Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2) document.

TABLE 2

Steps for developing instruments using a sequential mixed‐methods design

Step 1. Reviewed existing literature while thoughtfully observing the whole implementation intervention activities, implementation processes and participants' experiences Step 2. Interviewed individuals about their specific capacity building experiences Step 3. Conducted focus group interviews about collective capacity building experiences Step 4. Developed/adapted instruments in English while consulting implementation scientists Step 5. Revised the English language instruments based on feedback from CBO participants and CBPs Step 6. Pilot tested the English language instruments with two bilingual and bicultural individuals, using cognitive interviews Step 7. Revised the English language instruments based on the cognitive interview findings Step 8. Translated the English language instruments into Spanish and Arabic by professional translators Step 9. Checked translation accuracy and cross‐cultural equivalences of Arabic version by other professional translators and Spanish version by bicultural and bilingual individuals Step 10. Revised translated instruments while pilot‐testing with bilingual individual (i.e., Arabic) and bicultural individual (i.e., Spanish) using cognitive interviews Step 11. Finalized instruments in Arabic, English and Spanish

Abbreviations: CBO, community‐based organization; CBPs, capacity building providers.

TABLE 3

Demographics of the community‐based organization participants (n = 17 )

Characteristics	n	(%)
Role
Director	6	(35.3)
Staff (Full‐time)	7	(41.2)
Staff (Part‐time) & Volunteer	4	(23.5)
Highest level of schooling
Some college or technical school & Graduated college	7	(41.2)
Graduate school and beyond	9	(52.9)
No Answer	1	(5.9)
Ever developed a programme before?
No	4	(23.5)
Yes	11	(64.7)
No Answer	2	(11.8)
In general, would you say you (and your family living with you) have more money than you need, just enough for your needs, or not enough to meet your needs?
More money than needs	3	(17.6)
Just enough for needs	11	(64.8)
Not enough money	3	(17.6)
Perceived Net Finances. How much money would you have left over if you turned all your assets (Jewellery, car, house, etc.) into cash and paid off your bills?
Be in serious debt	6	(35.3)
Break even	4	(23.5)
Have money left over	6	(35.3)
No Answer	1	(5.9)
Birthplace
U.S.	8	(47.1)
Outside of the U.S.	9	(52.9)
Social groups
White	2	(11.8)
Immigrant, Black, Indigenous, People of Colour, Latinx or Asian	15	(88.2)
First language
English	8	(47.1)
Other language	9	(52.9)
Ease of Information/English fluency	Mean	(SD)
Reading the materials	3.29	(1.05)
Listening at the workshop	4.06	(0.90)
Speaking at the workshop	3.82	(0.81)
Writing the deliverables/assignments	3.53	(0.94)
Work Characteristics	Mean	(SD)
Length of time working at the organization, months (n = 16)	60.07	(56.44)
Years working with diverse communities (n = 17)	13.59	(10.20)
Total number of Initiative grants per CBO (n = 12)	4.25	(1.14)
How many languages do you speak? (n = 9)	2.11	(0.33)
Total number of languages CBOs' client use	28 different languages
Characteristics Related to Grant Work
Is the work related to capacity building added on top of your regular work that you are doing in the agency?	n	(%)
No	1	(5.9)
Yes	14	(82.3)
Somewhat	1	(5.9)
No Answer	1	(5.9)
Did you know you will have four capacity building providers?
No	15	(88.2)
Yes	1	(5.9)
No Answer	1	(5.9)
The grant would have a 6‐month Phase I to develop a new programme or practice?
No	3	(17.6)
Yes	14	(82.4)
Time it takes to create a meaningful, new programme (n = 10)	9.30	(3.97)
Time it takes to create a meaningful programme modification (n = 7)	5.86	(3.67)

Abbreviation: CBOs, community‐based organizations.

Among a total of 20 participants from nine CBOs, 17 participants from nine CBOs provided demographic data.

Participant sample

The study included 20 participants from nine CBO grantees who provided informed research consent. Inclusion criteria were individuals who: (1) worked or volunteered in the CBO; (2) received the Initiative's HBP intervention; (3) participated in a focus group, survey or individual interview; and (4) gave consent to use their information in research. Individuals from participating CBOs who did not give research permission were not included.

Data collection

Individuals from 10 CBOs received study information to review beforehand. Active recruitment occurred from February 2019 to November 2019. In March 2019, individuals from 10 CBOs attended an introductory recruitment session. One CBO declined participation after the introductory session. Thereafter, individuals were recruited into the study as they joined the Initiative's HBP intervention. Data were collected in March 2019 (focus group), April/May 2019 (T2) and November 2019 (T3). Data collected in April/May were listed as (T2) because no real time baseline data were collected before the intervention start date in November 2018 (T1). This article includes findings from April/May and November 2019 surveys that included quantitative instruments and qualitative responses to open‐ended questions. Focus group findings will be reported in another article (Parker et al., In review). Participants answered surveys in online or paper formats while in a group or individually at their selected place. All data were kept in university research offices and survey data were entered in the Research Electronic Data Capture (REDCap) system (Harris et al., 2009) at the university.

Data presented are from 20 participants: 17 participants from nine CBOs for the April/May survey and 17 participants from nine CBOs for the November survey. Fourteen out of 20 participants completed both April/May and November surveys. Individuals received $50 for each survey, interview, focus group or survey session regardless of their consent to use their data in research. Online Only Table 4 summarizes variables/instruments, sample items, answer choices, scores, data collection times and Cronbach's alphas. Face validity was checked with instruments. Online Only Table 5 summarizes the study concepts.

TABLE 4

Mean differences of capacity building among nine community‐based organizations (N = 9 CBOs )

Variables	April (T2) 2019		November (T3) 2019
	M	SD	M	SD	Change (M)	Change (SD)	t a	p‐value
Organizational context outcomes
Organizational strengths	4.11	.43	4.10	.56	−.01	.53	−.06	.96
Organizational conditions	3.36	.82	3.29	.82	−.07	.75	−.30	.77
Implementation process outcomes
Institutional procedural discrimination b	2.43	2.95	1.29	.99	−1.14	2.82	−1.51	.15
	May (T2) 2019		November (T3) 2019
Use of capacity building guiding equity principles	3.24	.84	3.22	.89	−.02	1.05	−.05	.96
	April (T2) 2019		May (T2) 2019
Ease of information	3.72	.66	3.88	.66	.15	.69	.67	.52

Note. All data analysis was done using organizational level data except with the Institutional procedural discrimination instrument.

CBOs = community‐based organizations. Paired t‐test results using organization as the unit of analysis (N = 9 CBO); The data represent nine CBOs (n = 17 participants) for the April/May survey and nine CBOs (n = 17 participants) for the November survey.

Paired t‐test results using the individual as the unit of analysis (n = 14 participants provided both T2 and T3 data).

TABLE 5

Changes in organizational capacity outcomes per answers from May to November 2019 (N = 9 CBOs )

Variables		Answers
15 capacity building areas (N = 9 a )		CBOs b already had this as desired before the project started	Not yet in place as CBOsb desired & Partially/almost in place as CBOsb desired	Completed but not as CBOsb desired	Fully in place as CBOsb desired	Total number of CBOsb
	May 2019 (T2)	1 c (2 d )	5 (6d)	1 c (2 d )	3 c	10 c (13 d )
	November 2019 (T3)	1 c (2 d )	4 c	1 c (2 d )	3c (4d)	9 c (12 d )
Culturally Responsive NIRN Practice Profile (N = 9 a )			Not yet in place as CBOsb desired; Not yet in place, process is going well; & Partially/ almost in place as CBOsb desired	Completed but not as CBOsb desired	Fully in place as CBOsb desired	Total number of CBOsb
	May 2019 (T2)		4c	2 c (3d)	2c (3d)	8c 10d
	November 2019 (T3)	e	4 c	1 c	4c (5d)	9 c (10 d )

Paired t‐test results using nine CBOs' (community‐based organizations) data; the unit of analysis was the CBO. The data were from nine CBOs with 17 participants for May and nine CBOs with 17 participants for November. Total sample size was 20 participants.

Wording in the survey was ‘we’, meaning CBOs.

Rounded at 0.5 level.

Rounded at 0.1 level.

Not an answer choice.

Organizational context outcomes

Organizational context outcomes included organizational strengths and organizational conditions.

Organizational strengths

This instrument was adapted from Organizational Readiness for Implementing Change (Shea et al., 2014) and Organizational Resilience (Kantur & Iseri‐Say, 2015) instruments. This 26‐item, 5‐point Likert Scale instrument has five subscales that measure comfort, resources, commitment, confidence and resilience. A sample item was, ‘We want to implement this program’. The total mean score was used. Higher scores indicated higher organizational strengths. Study reliability using Cronbach's alpha was .92.

Organizational conditions

This 13‐item, 5‐point Likert Scale instrument includes seven items about workload manageability, frequency of people turn‐over, communication in the organization, communication with funders, partnerships with families and communities, satisfaction with organizational systems and practices and executive leadership support. It also includes six items about sustainability planning. The items were developed using focus group findings that asked participants' desired outcomes after completing 2 years of capacity building (Parker et al., In review). A sample item was, ‘How manageable is your workload now?’ The total mean score was used. Higher scores indicated better work environments. Study Cronbach's alpha for sustainability was .93.

Implementation processes outcomes

Implementation processes outcomes included institutional procedural discrimination, capacity building guiding equity principles and ease of information.

Institutional procedural discrimination

This 12‐item, binary (yes/no) instrument asks about participants' experiences of institutional procedural discrimination during the Initiative's HBP intervention (Lim et al., 2022). The first four survey items measured experiences such as changing procedures without permission, having meetings and materials based on Western culture, words on documents being changed without explanation and cultural ways not being respected when they were known. A sample survey item was, ‘All meetings and materials were based on Western culture, so that I had difficulty understanding the work’. The last eight items measured being treated differently due to racial identity, nationality, English as a second language, gender, religious identity, age, sexual orientation or income. The mean score of the number of ‘yes’ responses was used. Higher scores indicated that participants experienced more discrimination. Study reliability using Cronbach's alpha was .96.

Capacity building guiding equity principles

This 10‐item, 5‐point Likert Scale instrument asks about the use of 10 capacity building guiding equity principles in the overall Initiative's HBP intervention process (Wang et al., In preparation). The principles were developed by the funding staff and CBPs, with facilitators and researchers present. The principles included racial equity, cultural responsiveness, intersectionality, transformational, trauma informed, sustainability, continuous learning, relationship based, information informed and contextual responsiveness. Before data collection, participants received the principles defined in multiple languages including Arabic, English and Spanish. The Somali translation was not requested when asked. The surveys also included short description of each principle. A sample principle was, ‘Racial Equity: Prioritizes resources, voice, and access to power to address the root causes of racial inequities’. The total mean score was used. Higher scores indicated more use of the capacity building guiding equity principles in the whole Initiative's HBP intervention process. Study reliability using Cronbach's alpha was .96.

Ease of information

This 4‐item, 5‐point Likert Scale instrument measures the ease of reading the materials, listening at the workshops, speaking at the workshops and writing deliverables/assignments. Ease of information was measured in April 2019 and May 2019. A sample item was, ‘How much were you able to do the following with ease? Read the materials’. The total mean score was used. Higher scores indicated easier information access. Study reliability using Cronbach's alpha was .86.

Organizational capacity outcomes

Organizational capacity outcomes include the 15 capacity building areas, progress in the Culturally Responsive NIRN Practice Profile, theory of change, use of the literature review abstracts and organizations better off from capacity building data.

Fifteen capacity building areas

This instrument was adapted from the Implementation Drivers Assessment (Fixsen et al., 2018) developed by the NIRN. It measures participants' perceptions about the 15 capacity building areas including deliverables, such as: (1) a method of writing about programmes, (2) programmes developed as desired, (3) engaging families when developing programmes, (4) informing families about the programmes, (5) recruiting and hiring staff, (6) staff training, (7) measuring staff performance, (8) an evaluation plan method, (9) an evaluation plan for the home‐based programmes, (10) a racial equity theory of change, (11) a racial equity theory of change for home‐based programmes, (12) a culturally responsive NIRN practice profile, (13) a 2‐year budget plan, (14) seeking help from CBPs for what is needed and (15) seeking help from CBPs for ways to culturally adapt the home‐based programme. A sample item was, ‘Our organization has a racial equity theory of change’. Answer choices were: (1) we already had this as desired before the project started, (2) not yet in place as we desire, (3) partially/almost in place as we desire, (4) fully in place as we desire and (5) completed but not as we desired. The number of organizations were calculated for each item to report change.

Progress in the culturally responsive NIRN practice profile

This single item asks about CBOs' progress in developing a home‐based programme protocol, specifically the Culturally Responsive NIRN Practice Profile. The question was, where is your organization in the process of developing a well‐defined programme? Answer choices were organized into: (1) not yet in place as we desire, (2) not yet in place, process is going well, (3) partially/almost in place as we desire, (4) completed but not as we desired and (5) fully in place as we desire. The number of organizations were calculated for each item to measure change.

Theory of change

This 12‐item, 5‐point Likert Scale instrument asks how well the short‐term and intermediate‐term goals of the theory of change developed for the Culturally Responsive NIRN Practice Profile Well‐Defined Programme were used in the whole Initiative's HBP process. Researchers developed this instrument using the items in the theory of change created by the funding staff and a CBP team. Participants rated the theory of change, rated related conditions in May 2019 (T2) and retrospectively rated baseline conditions. The theory of change was also rated in November 2019 (T3). A sample item was, ‘My organization currently designed a program integrating community knowledge and input with [name of the CBP team]. The total mean scores for the short‐term and intermediate‐term goals were used. Higher scores indicated more progress. Study reliability using Cronbach's alpha was .89 for short‐term goals and .90 for intermediate‐term goals.

Use of the literature review abstracts

The single item, ‘Please tell us about your use of the literature review,’ ascertains information about the use of the home‐based programme literature review abstracts. The answer choices were: (1) not in yet, (2) not yet in place, process is going well, (3) not yet in place as we desire, (4) partially/almost in place as we desire, (5) completed but not as we desired and (6) fully in place as we desire. The response indicated the usage. Participants also answered the open‐ended prompt, ‘Please tell us more about your experience using the literature review’.

Organizations better off from capacity building

Participants provided responses using the open‐ended prompt, ‘Please provide 3 examples of how your organization is better off as a result of capacity building since the launch of the work in November of 2018’.

Workshop outcomes

The well‐defined programme CBP team and staff developed these questions with facilitation support from researchers. inquires about the contents of the NIRN model using learning outcomes for each workshop. The beginning stem of a sample item was, ‘As a result of the workshop, my knowledge about how to develop a practice profile and why it matters…’ The answer choices were: (1) increased, (2) stayed the same and (3) decreased/I got confused. rates participants' satisfaction using a 4‐item, 5‐point‐Likert Scale. A sample item was, ‘I was satisfied with the workshops overall’. Study reliability using Cronbach's alpha was .86. asks about the use of resources provided using four‐items. A sample item was, ‘After all workshops I attended, I was able to increase my learning by completing the Learning Assignments outlined’. Answer choices were: (1) yes, (2) no and (3) somewhat.

Ethical considerations

The university's institutional review board approved the study. Participants provided written informed consent using the English language version, although the consent forms were also prepared in Arabic, English, Somali and Spanish. Subsequent to asking the entire group, some participants volunteered to receive real time data reports to validate findings. After those participants' review, real time data reports were also submitted to the participants and the funders. The data were stored at the university.

Data analysis

After inputting the data in REDCap, two different team members verified data entry accuracy and a third team member checked specific entries again. Data integrity was reviewed again months later. Blank entries in the surveys were identified as such without any imputation. Quantitative data were analysed using SPSS (version 25) for Windows. Descriptive statistics such as means, standard deviations, frequencies and percentages were calculated for demographics and other study variables. The research question was examined using means and paired t‐tests, with the organization as the unit of analysis. For organizational level data, mean scores of participants were calculated (ranged from 1 to 3 individuals in each CBO). One exception was the institutional procedural discrimination instrument, which were analysed using paired t‐tests, because the survey questions asked about individual experiences. Workshop outcomes were analysed using descriptive statistics. Qualitative data were inserted into a Word document and analysed using inductive content analysis. The process included: (1) identifying units of analysis, (2) reviewing the whole data set, (3) coding data by categories/themes, (4) grouping categories/themes and (5) finalizing categories/themes and exemplar quotes (Prior, 2020). For use of the literature review abstracts, answer options were matched with written comments.

Validity and reliability/rigour

This study used a sequential exploratory, mixed‐method design to develop, organize and process qualitative and quantitative data. The goals of instrument development and adaption were centred on decreasing the number of items, fitting the specific Initiative's HBP intervention, and fitting the multicultural, multilingual participants. As depicted in Table 2, the researchers used 11 instrument development steps. The first three steps were conducted to develop and organize qualitative data. In step four, the English language instruments were developed/adapted using qualitative focus group study findings (Parker et al., In review). Researchers developed several instruments by engaging study participants to ensure instrument validity. For example, during individual interviews participants described experiences of being discriminated against. Researchers used this information to develop the institutional procedural discrimination instrument. Detailed information about institutional procedural discrimination is reported in another article (Lim et al., 2022). Most instruments asked about participants' perspectives about their CBOs, except the institutional procedural discrimination and ease of information instruments, which asked about individual experiences. Instruments also included both open‐ and close‐ended surveys per participants' suggestions.

In steps five to seven, the content of the English language instruments was checked with an implementation scientist, participants' feedback and cognitive interviews. Cognitive interviewing is a technique used to verify validity, meaning to determine if concepts or items were understood by participants as researchers intended (U.S. Department of Health and Human Services, 2009; Howlett et al., 2018). In steps eight to ten, the finalized English language instruments were translated into Spanish and Arabic, and cross‐cultural equivalences were checked. Participants did not request translation into Somali. Lastly, in step 11, instruments were finalized in Arabic, English and Spanish. Study reliabilities using Cronbach's alphas of these developed or adapted instruments ranged from .86 to .96.

RESULTS/FINDINGS

Demographic characteristics

As indicated in Table 3, participants included directors, staff and a volunteer from nine CBOs. Most participants (52.9%, n = 9) had graduate school and beyond education (41%, n = 7) had graduated college or had some college or technical school, and one participant declined to answer. Slightly more than half (n = 9) of the participants were born outside of the U.S., and English was not their first language. Participants (76.5%, n = 13) identified as Immigrant, Black, Indigenous, People of Colour, Latinx or Asian. Participants reported ease of listening (4.06 ± 0.90, score 4 = well) and speaking (3.82 ± 0.81, score 3 = neutral) at the workshops. On average, participants had worked at their CBOs for about 6 years, worked with diverse communities for 14 years, spoke two languages and served families speaking 28 different languages, such as Arabic, English, Somali and Spanish. Most (64.7%, n = 11) had previous programme development experience. Fourteen out of 17 (82.3%) participants developed home‐based programmes in addition to their regular workload.

Correlations among study variables

The sample size for the correlational data was based on nine CBOs and the unit of analysis was at the organizational level. Ease of information was correlated with total organizational strengths in April (r = .81, p < .05) and November (r = .72, p < .05). Ease of information was also correlated with total organizational conditions (r = .92, p < .01) in April. Average years of serving diverse communities were correlated with (r = .68, p < .05) institutional procedural discrimination in April.

Comparison of capacity building outcomes between April and November

Table 4 depicts organizational level means and standard deviations as well as t‐test results for organizational contexts and implementation processes outcomes in April/May (T2) and November (T3).

Organizational context outcomes

The mean of organizational strengths was slightly over 4 points, and the mean of organizational conditions was slightly over 3 points in both April/May and November. In the paired t‐test, no significant mean changes were noted in any of the context outcomes.

Implementation processes outcomes

Study participants reported experiencing 2.43 types of institutional procedural discrimination in April 2019 and 1.29 types in November 2019. Fifteen out of 17 (88%) participants reported experiencing at least one of the 12 institutional procedural discrimination types in April, compared with 14 (82%) participants in November. Use of the capacity building guiding equity principles in the whole process was rated slightly over 3, meaning neutral, at T2 and T3. The overall ease of information mean at the CBO level was 3.72 in April and 3.88 in May of 2019. In the paired t‐test, no significant mean changes were noted in any of the implementation process outcomes.

Organizational capacity outcomes

Seven Culturally Responsive NIRN Practice Profile programmes were new, and two were modified. Table 5 depicts 6 to 7 month changes in progress in the 15 capacity building areas and Culturally Responsive NIRN Practice Profile. Each number represents the number of organizations reporting. The number of participants from the nine CBOs ranged between one and three. The answers were aggregated when there were 2 or 3 participants in the same CBO. When the answers of these two or three participants in the same CBOs did not match, the researchers developed data aggregation principles. For example, when there were two participants with different answers, each answer was given ½ point. When there were three participants, 1/3 point was given for each answer. Then, numbers were added and rounded to calculate the whole number of CBOs for each answer choice because there was no 1/3 or ½ CBO. In Table 5, the superscript letter c shows when the number was rounded at the 0.5 level; The superscript letter d indicates when the number was rounded at the 0.1 level. Rounding, in some cases, made the total number of CBOs greater than the sample size of nine CBOs.

About the progress in the 15 capacity building areas, 3c CBOs had fully developed capacities as desired in May, which stayed similar to 3c(4d) CBOs in November. For the Culturally Responsive NIRN Practice Profile, participants from the CBOs completed home‐based programmes not as desired; 2c(3d) CBOs did so in May and 1c did so in November. Nine completed the Culturally Responsive NIRN Practice Profile for the contract deliverable on 4‐30‐2019. However, in November, participants continued to revise the Culturally Responsive NIRN Practice Profile for implementation to develop specific home visiting, culture and language content; to adjust the home‐based programme for the new staff; or to meet required enrolment deliverables. In such cases, participants checked ‘not yet in place as we desired, and process is going well’ or ‘completed but not as desired’.

Table 6 summarizes the mean differences in the theory of change. No significant mean differences were noted when comparing T1and T2, and T1 and T3.

TABLE 6

Mean changes in the theory of change for nine community‐based organizations (N = 9 CBOs)

Characteristics	November 2018 a (T1)		May 2019 (T2)		November 2019 (T3)		T1 vs. T2	T1 vs. T3
	M	SD	M	SD	M	SD	t	t
Theory of change
Short‐term goals	4.03	.58	4.23	.52	3.77	1.25	.95	−1.20
Intermediate‐term goals	4.23	.54	4.43	.44	3.96	1.00	1.03	−1.35

Note. Data were collected from a total of 20 participants.

In the May 2019 survey, participants were asked to retrospectively rate the Theory of change because it was not developed until Jan./Feb. 2019.

Paired t‐test results using nine CBOs' data; the unit of analysis was the CBO. The data represent responses from nine CBOs (n = 17 participants) for the April/May data and nine CBOs (n = 17 participants) for the November data.

Table 7 summarizes 14 participants' answers and comments about their use of the home‐based programme literature review abstracts provided in June 2019. It was used by six participants. Five participants who used it noted that the literature review was provided after the Culturally Responsive NIRN Practice Profile was due (due on 30‐4‐2019) and submitted, or it was not in an easy‐to‐use format. For example, the information was in an Excel table, only the article abstracts were provided, and participants had to read the English language abstracts more than once to understand.

TABLE 7

Use of the literature review abstracts by participants' answers and comments (n = 14 out of 17)

Answer choices	Written comments
Not in yet (n = 4)	‘Was explained to use, not yet in as part of our training manual’. ‘I have not dedicated the time to look through it. Now that all staff has been hired and trained, I am hoping to get to that’.
Not in place as we desire (n = 1)	‘We have not utilized the literature review yet. It is unusual to have a general review for our program. We have never experienced anything like that’.
Completed but not as we desired (n = 1)	There were no written comments.
Fully in place as we desire (n = 5)	‘I did not find the literature review to be timely or relevant. I would have found it useful to have a review of successful home visiting programs earlier in our program planning to select components and features such as large group socializations and small clusters and parent networks. A review of effective staff recruitment, training and retention would also have been applicable when we were writing the practice profile. I did site sources from the literature review into our practice profile just as an exercise but their relevance was low’.
No response was checked in the survey but comments were written in the open box area (n = 3)	‘N/A: We did not use the literature review. Although we requested it many times from cap. [cap. means capacity] building consultant, it was not received until we were far enough into the implementation phase that 1. We were busy with new phase two obligations like reports and enrollment and 2. were already past a point that it could be meaningfully used or inform anytime we were doing. We/I was very disappointed with the lit. [lit. means literature] Review as we had many areas of interest that were not able to request (due to constraints of the assigned scope/purpose) and it felt like a waste of time for everyone, esp. [esp. means especially] [name of the maternal child consultant removed] since it wasn't shared with programs in a timely or relevant way’.

Note. Direct quotes were written in US English. Brackets [] were used to insert words or remove identifying information.

Table 8 summarizes nine participants' comments about how their CBOs were better off from capacity building. Themes included (1) programmes developed, (2) partnerships with CBPs, (3) implementation knowledge and (4) networking.

TABLE 8

Participants' reports of how organizations are better off from capacity building (n = 9 out of 17)

Themes	Example quotes
Programs developed	‘We are thinking more strategically about program development. We have tools that help guide our practice, that can be used to assess and reflect on how and why we are running this program’. ‘Program is developed!’
Partnerships with capacity building providers	‘The program staff and director meet w/[w/ means with] consultants and together develop tools to track data, make forms, design training appropriate to the needs of families. As a result the program teams feel invested and get excited about the program also awareness & experience on what capacity building is comes to life real people supporting us at [name of the organization was removed]’. ‘We are pleased to work with [the names of the capacity builders removed]. In June [the names of the capacity builders removed] came to our organization to offer a supervisory training and has meet with us one‐on‐one to address some staffing challenges. We did not know there was an agency designed to offer organizational capacity building support to community based organizations of color and it has been great to work with them’.
Implementation knowledge	‘Understand how to develop an Implementation Plan’. ‘*Better understanding of [the funder's name removed] expectations to develop – RETOC – [RETOC means racial equity theory of change] training manual etc.’ ‘We understand the next steps for Phase II’.
Networking	‘Increased networking/community‐building with other programs/orgs [orgs. means organizations]. Because of connections made during capacity‐building phase and workshops’.

Note. Direct quotes were written in US English. The asterisk * was written as part of the direct quote. Brackets [] were used to insert words or remove identifying information.

Workshops

Workshop attendance and hours spent completing homework

Participants attended 5.35 ± 1.0 workshops (89.2%, range 2–6 workshops) and spent an average of 18.85 ± 11.26 hours (range 4.67–37.50) doing homework that included contract deliverables after monthly workshops.

Overall implementation knowledge

Overall, 59% of participants reported their knowledge increased, 38% stayed the same and 3% decreased/I got confused. The topics that participants reported as ‘decreased/I got confused’ included the concepts about implementation science (n = 3), the Culturally Responsive NIRN Practice Profile (n = 2 to 3), the 2‐year budget (n = 1) and the racial equity theory of change (n = 1).

Overall workshop satisfaction

The mean workshop satisfaction was 3.84 ± 0.63, indicating between neutral (score 3) and agreeing (score 4) with satisfaction. Participants reported gaining new content knowledge (4.06 ± 0.68) the highest, followed by trusting the information (3.87 ± 0.72), finding the information relevant (3.75 ± 0.86) and being satisfied with the workshops overall (3.69 ± 0.70).

Overall resource use

Sixty percent of participants (n = 17) reported that they increased learning by using provided resources, followed by 29% reported somewhat and 12% reported no. The most helpful resource was reaching out to my assigned CBP with questions (74%), followed by using the Dropbox for materials I may have missed (72%), completing the learning assignments outlined (56%) and reading the articles or watching the videos referenced (38%). Sixty percent of participants reported that they easily used the resources and 40% reported it was not easy. The easiest resource to use was the Dropbox (100%), followed by reaching out to the CBP (59%), reading the articles or watching the videos (50%) and completing the learning assignments (44%).

DISCUSSION

This longitudinal study using community‐based participatory research principles examined the impact of the Initiative's HBP intervention on organizational contexts, implementation processes and organizational capacity outcomes (Table 1). This study provided new knowledge about how to use a sequential mixed‐method design and multiple instruments across three languages. We highlighted participants' voice first by gathering qualitative data, and then verified it with quantitative findings. This study also shows how to engage participants who practice multiple languages and cultures to co‐design sensitive research about bicultural experiences. Qualitative findings from a prior focus group (Parker et al., In review) were incorporated into this discussion section to highlight current survey findings.

Participants reported excellent individual and CBO strengths in April and November 2019. For example, the 20 participants had excellent educational, programme development experience and multicultural, multilingual capabilities. The total mean score of organizational strengths (4.11 ± 0.43) was higher than what Sharma et al. (2018) previously found (i.e., 3.54–3.64) (Sharma et al., 2018); it was similar to what Damschroder et al. (2021) found (i.e., 3.93–4.22) (Damschroder et al., 2021). Higher organizational strengths indicated that these participants were psychologically and behaviourally prepared to implement organizational change (Weiner et al., 2008). They also worked in stable CBOs because no change was noted in their organizational conditions.

In May and November 2019, participants reported that capacity building guiding equity principles were used slightly more than fairly well (3.22–3.24) in the whole process, with three meaning fairly well and four meaning quite well. Participants working at CBOs with existing organizational strengths perceived that the principles were used well in the whole process. Principles were not reported as used well for CBOs with less existing organizational strengths. Aligning policy and programming with guiding principles were noted as an effective strategy for implementing evidence‐based programmes equitably (Bryson et al., 2017; Pérez‐Escamilla, 2018). Correlations of the capacity building guiding equity principles with the organizational contexts is reported elsewhere (Wang et al., In preparation).

From April to November 2019, the mean scores of institutional procedural discrimination decreased, although it was not significant. In April 88% of the multilingual, multicultural participants and 82% in November continued to experience at least one of the 12 types of institutional procedural discrimination. Participants with more experience with diverse communities reported more types of institutional procedural discrimination, indicating that the overall Initiative's HBP intervention privileged the funder's ways of knowing and doing. Participants' multicultural, multilingual backgrounds and work with families enabled them to get funding. However, after receiving funding, those backgrounds became points of discrimination (Lim et al., 2022).

Some CBOs were still in need of equitable capacity building after the Initiative's HBP intervention. The Initiative's HBP intervention was intended to support equity, meaning those CBOs with less organizational strengths; yet, the design further supported those CBOs with more strengths (Wang et al., In preparation). As workshop outcomes, CBPs increased participants' knowledge and were reported as the most helpful resource. However, those positive aspects were not enough to change organizational capacity outcomes in the Initiative's timeline.

The timeline was enough for participants to increase implementation knowledge, which is consistent with a previous finding (Mosson et al., 2019). However, some participants were confused by the Culturally Responsive NIRN Practice Profile, the model required to develop the home‐based programme by the CBOs. Other findings indicated that some participants checked the ‘box’ to complete deliverables (Parker et al., In review), which is inconsistent with a community‐designed approach. These conditions may explain no statistically significant increase in organizational capacity outcomes, including the 15 capacity building areas, the Culturally Responsive NIRN Practice Profile and the theory of change.

The Initiative's timeline was for 6 months to develop the Culturally Responsive NIRN Practice Profile (December to April). Three contract extensions were needed for other deliverables all due in May because the Initiative's timeline was based on a Western, native English speaker working full‐time for pay (Parker et al., In review). Participants' regular and contract work continuously accumulated. Not giving enough time to develop a useable home‐based programme can be viewed as institutional procedural discrimination (Lim et al., 2022). So, it is not surprising that there were no statistical changes in implementation processes outcomes.

More reasons may explain no significant statistical changes and highlight the Initiative's factors to address prior to designing a home‐based programme. First, there was no opportunity for baseline data collection before CBOs started the Initiative's HBP intervention because participants were learning expectations and finalizing teams 2 months after starting the Initiative's HBP intervention (February 2019). Baseline data were previously recommended to provide a feasible level of expected change (DeCorby‐Watson et al., 2018). In this study, no baseline data resulted in higher scores on some instruments in May 2019 (after completing 6 months of workshops). For example, in May 2c (3d) CBOs reported having the Culturally Responsive NIRN Practice Profile fully in place as they desired.

Second, CBOs and CBPs were not engaged in selecting the NIRN model, processes, outcomes or timelines. Additionally, the NIRN model facilitators and barriers were not examined. In a focus group with CBOs, participants reported wanting the funder to understand their backgrounds, culture and language (Parker et al., In review). About 88% of participants identified as Immigrant, Black, Indigenous, People of Colour, Latinx or Asian (Table 2). Involvement in the decision‐making process is an essential component in perceiving interactions and procedures as just (Kunard & Moe, 2015). Study participants were required to use the NIRN model to develop their home‐based programmes and they reported that the use of the mainstream Western implementation model and scientific process for capacity building did not fit their thinking processes (Parker et al., In review; Lim et al., 2022). This resulted in more participants reporting that the meetings and materials presented were based on Western culture, so more people had difficulty understanding the work in November (65%, n = 11) than in April (47%, n = 8) (Lim et al., 2022). These findings are consistent with a previous study that the NIRN model provided a programme implementation framework, however, it was very complex, resource intensive and the language developed for the educational system did not fit well in healthcare (Brémault‐Phillips et al., 2018).

Study participants wanted to develop community‐designed, home‐based programmes. The NIRN model could be used to identify key implementation processes, facilitators and barriers as well as to develop a model adaptable for local communities (Bryson et al., 2017; Fearing et al., 2014; Lala et al., 2016; Wolfe et al., 2019), but that was not done in this intervention with these participants. Effective evidence‐based, home‐based programme development requires a deep understanding of culture, community and service contexts as well as extensive knowledge about community needs and home‐based programme adaptation processes (Hiratsuka et al., 2018).

Third, the Initiative's HBP intervention timeline did not fit participants' timeline for home‐based programme development or modifications, and participants reported that the process was moving too fast (Parker et al., In review). A previous implementation collaboration among public health, clinical partners, community and academia took 1 year for conceptualization (pre‐funding), 1 year to adapt a programme, and 1 year to implement the programme (Brock et al., 2019). Participants had 6 months to learn and integrate implementation concepts and concurrently develop a community‐designed home‐based programme with client input. With their previous programme development experiences, they reported that it would take 9 months to develop a meaningful new programme or 6 months to make meaningful programme modifications to an existing programme (Table 2). They were not asked by the funders or researchers how long it would take for them to learn and integrate implementation concepts or gather client input. In a focus group, eight out of nine organizations agreed that time was a concern (Parker et al., In review). Related to time, some participants wanted more time to digest what the NIRN model tools would look like in practice (Parker et al., In review). Especially, when working with different cultures and languages, which necessarily takes more time and care, participants wanted to do the work well (Parker et al., In review; Wang et al., In preparation).

Fourth, participants were given the home‐based programme literature review abstracts in June 2019, after the Culturally Responsive NIRN Practice Profile was submitted in April. In a March 2019 focus group, participants asked for more direct examples of end results (Parker et al., In review). Similarly, in the survey, participants reported that the literature review abstracts were not helpful in developing the programme because the abstracts were delivered after the Culturally Responsive NIRN Practice Profile was due. An alternative process might have been the one used by the federal home visiting initiative, which funded four urban Indian communities and tribal government to implement culturally tailored home‐based programmes (Hiratsuka et al., 2018). Researchers worked with those communities to make cultural adaptations of evidence‐based home‐visiting programmes through careful community needs assessments and modifications to fit the community (Esposito et al., 2014; Hiratsuka et al., 2018). In this Initiative's HBP intervention, participants did not receive similar direct support to develop the essential components of the home‐based programme.

Fifth, adequate, planful and paid time was needed to develop multicultural, multilingual programmes. When an author called some participants to clarify the November survey responses, participants reported revising the Culturally Responsive NIRN Practice Profile, so that they checked ‘not yet in place as we [CBOs] desired, process is going well’ or ‘completed but not as desired’. This indicated that participants submitted the programme in April 2019, which they developed not as desired, and were revising it for desirability in November 2019. For example, some developed the home‐based programmes in English for the contract and then had to translate the Culturally Responsive NIRN Practice Profile into different languages and complete cultural tailoring. Nine CBOs served clients with 28 different languages. In April, participants reported that they were working (82.3%, n = 14) or somewhat working (5.9%, n = 1) unpaid ‘on top of’ their regular work. In November 2019, as they made the programmes multilingual and multicultural, participants continued to work unpaid ‘on top of’ their regular work (Lim et al., 2022).

Four concepts related to being better off from capacity building included programme development, partnerships with CBPs, implementation knowledge and networking. These findings are consistent with a previous study that participants valued partnerships with CBPs and networking as positive aspects of capacity building (Parker, In review). Other positive aspects reported in November 2019 were participants increased knowledge about how to develop an implementation plan, a racial equity theory of change and a training manual. These findings were important to consider given that an implementation plan (original due date 31‐5‐2019, extended due date 6‐2019) and a racial equity theory of change (due in 28‐2‐2019) were submitted as contract deliverables. CBPs helped participants complete their contracts and ongoing work. Knowledge about the implementation plan and racial equity theory of change could benefit from knowledge of data systems, organizational capacity and programme development simultaneously. Introducing supportive environment CBPs, data systems CBPs and organizational capacity CBPs while developing the Culturally Responsive NIRN Practice Profile might bring these necessary components together earlier if given an appropriate timeline.

Strengths and limitations

This study had multiple strengths. First, this is a methodologically well‐designed study that used the organization as the unit of analysis, filling a gap in implementation science (DeCorby‐Watson et al., 2018). Second, the organizational sample size was enough to have 32% power for an effect size of .5 and 85% power for detecting an effect size of 1.0. Third, the instruments were primarily developed by working with participants and CBPs helped develop the theory of change, which ensured relevant data collection about what mattered for them. Thus, the instruments had high internal consistency as measured by Cronbach's alphas (.86–.96). By using a sequential exploratory mixed‐methods, researchers were able to discuss important contributions to knowledge development and possible reasons for statistically non‐significant study findings. Fourth, we used multiple innovative implementation science outcomes. This resulted in developing tools, which captured what participants were experiencing. Fifth, the longitudinal study data helped to detect changes or no changes over time. Sixth, lessons learned can help future health initiatives to develop appropriately funded culturally and linguistically relevant programmes for multicultural, multilingual participants.

There were limitations. First, among the 20 participants, 14 provided T2 and T3 data. Although 14 participants were the same, the change of six participants might have threatened internal validity (Torre & Picho, 2016). This may also be related to no changes in organizational capacity outcomes; some who learned implementation knowledge from the workshops were no longer at the CBO. Second, no one withdrew from the study despite having the matched sample size decrease. Also, when only one participant from a CBO was involved, this person had the sole burden of representing the CBO. Finally, these study findings are embedded in unique contextual factors; the findings need to be understood as a specific example, rather than being generalized. If researchers want to replicate this study, they can check suitability of the current findings using the Assessment for Applicability and Transferability of Evidence resource (Buffet et al., 2011).

CONCLUSION

These findings can inform how to promote equity practices in participatory research and implementation science in other settings and/or countries. Excellent strengths among individuals and CBOs can flourish when public initiatives involve participants in selecting an appropriate implementation science model, intervention timeline and processes. Fully incorporating participants' design ideas as well as thinking and timing processes can help develop culturally and linguistically appropriate home‐based programmes. Building a clear contract with appropriate budget allocations, expectations communicated and consistent relationships between initiative staff and participants will help minimize discrepancies between the Initiative's programme contract deliverables and what is needed.

Nurse researchers can study participants' barriers and facilitators, cultural strengths, needs and aspirations. They can also work with stakeholders in the CBOs to determine baseline data collection times, engagement strategies and appropriate implementation science models. Including nurse clinicians in home‐based programme development can support baseline assessments before interventions. Nurse clinicians can also support timely delivery of evidence‐based literature so that participants can develop or adapt interventions to fit CBOs. Nurse educators can use study findings to teach how to budget for and develop strengths‐based approaches, and collaborative timelines for working with participants. Future studies can include exit interviews with participants to describe their experiences after the implementation of the Culturally Responsive NIRN Practice Profile and their sustainability plans. The study instruments can be useful in research or practice with similar multicultural, multilingual CBOs.

AUTHOR CONTRIBUTIONS

Dr. Eunjung Kim designed the study as a parenting education expert working with multicultural, multilingual community organizations, led article conceptualization, led the literature review, co‐led the methods, data analysis, results, discussion, conclusion and provided substantive research knowledge. Dr. Doris M. Boutain as a community based participatory researcher designed the study, co‐led socially just community engagement and implementation research, co‐led the methods, data analysis, results, discussion, conclusion and provided substantive research knowledge. Mr. Sungwon Lim co‐led methods, including data collection, verification and analysis, and provided substantive research knowledge. Ms. Sanithia Parker co‐led the literature review on community‐based organizational contexts, data collection, added discussion content using equity expertise, conclusion and provided substantive research knowledge. Ms. Di Wang co‐led methods, including data collection and verification and provided substantive research knowledge. Dr. Rebekah Maldonado Nofziger led community‐based engagement, data collection, verification, archiving and provided substantive research knowledge. Dr. Bryan J. Weiner was consulted for instrument development and provided substantive research knowledge. Real time data dissemination and analysis occurred with participants, capacity building providers, funding staff, and the primary funder. No community partners were included as authors because some author invitees declined to protect participants, CBOs, community funding, and safety.

CONFLICT OF INTEREST

Authors have no conflict of interest related to this research article.

PEER REVIEW

The peer review history for this article is available at https://publons.com/publon/10.1111/jan.15276.

Tables S1‐S5

Click here for additional data file.