Continuous quality improvement intervention for adolescent and young adult HIV testing services in Kenya improves HIV knowledge.

OBJECTIVES: To determine whether continuous quality improvement (CQI) improves quality of HIV testing services for adolescents and young adults (AYA).
DESIGN: CQI was introduced at two HIV testing settings: Youth Centre and Voluntary Counseling and Testing (VCT) Center, at a national referral hospital in Nairobi, Kenya.
METHODS: Primary outcomes were AYA satisfaction with HIV testing services, intent to return, and accurate HIV prevention and transmission knowledge. Healthcare worker (HCW) satisfaction assessed staff morale. T tests and interrupted time series analysis using Prais-Winsten regression and generalized estimating equations accounting for temporal trends and autocorrelation were conducted.
RESULTS: There were 172 AYA (Youth Centre = 109, VCT = 63) during 6 baseline weeks and 702 (Youth Centre = 454, VCT = 248) during 24 intervention weeks. CQI was associated with an immediate increase in the proportion of AYA with accurate knowledge of HIV transmission at Youth Centre: 18 vs. 63% [adjusted risk difference (aRD) 0.42,95% confidence interval (CI) 0.21 to 0.63], and a trend at VCT: 38 vs. 72% (aRD 0.30, 95% CI -0.04 to 0.63). CQI was associated with an increase in the proportion of AYA with accurate HIV prevention knowledge in VCT: 46 vs. 61% (aRD 0.39, 95% CI 0.02-0.76), but not Youth Centre (P = 0.759). In VCT, CQI showed a trend towards increased intent to retest (4.0 vs. 4.3; aRD 0.78, 95% CI -0.11 to 1.67), but not at Youth Centre (P = 0.19). CQI was not associated with changes in AYA satisfaction, which was high during baseline and intervention at both clinics (P = 0.384, P = 0.755). HCW satisfaction remained high during intervention and baseline (P = 0.746).
CONCLUSION: CQI improved AYA knowledge and did not negatively impact HCW satisfaction. Quality improvement interventions may be useful to improve adolescent-friendly service delivery.

Introduction

Adolescents have the highest HIV incidence of any age group and are the only age group in which HIV-related mortality increased between 2005 and 2013 [1]. While substantial progress has been made in HIV testing and treatment for adult populations, there has been less programmatic focus on adolescents (aged 10–19 years) and young adults (aged 20–24 years) [2,3]. Eighty-three per cent of all adolescents living with HIV reside in sub-Saharan Africa (SSA), and yet just 9–13% of adolescent boys and girls in the region have tested for HIV in the past year [4-6].

Addressing adolescents and young adults’ (AYA) unique HIV needs requires effectively promoting HIV prevention behaviors, and also detecting and managing HIV infection early by building routine HIV testing habits. Healthcare workers (HCWs), however, report feeling uncomfortable providing services to AYA. Major concerns highlighted by HCW include lack of confidence in counseling and interpersonal communication [7]; AYA depression and mental health; concern about doing the wrong thing; and challenges handling disclosure, AYA autonomy, and consent [8].

Whereas some studies have focused on uptake of HIV testing among AYA populations [9], fewer have focused on improving the perceived quality of such services. Quality services, defined as being well tolerated, effective, patient-centered, timely, efficient, and equitable [10], are an essential component of the World Health Organization's (WHO) adolescent-friendly strategy [11,12]. Quality interventions not only result in improved clinical practice [13], but have also demonstrated results in patient outcomes, including HIV prevention behaviors [14].

Continuous quality improvement (CQI) is a method of process optimization that includes testing a series of small, iterative changes for impact on a system. It is a flexible process that is particularly useful in healthcare settings because it can address unique system challenges and test locally informed solutions. Employing CQI and directly measuring and reviewing impact on adolescents’ perceptions of service quality may provide an acceptable and low-risk method for service delivery improvement. CQI has a large body of evidence, supporting its effectiveness in creating sustainable improvements in healthcare delivery in both resource-rich and resource-limited countries [13,15-19].

This study – Developing Adolescent Strategies for HIV Testing (DASH) – focused on improving the quality of AYA HIV testing services (HTS) at a large referral hospital in Kenya by conducting a CQI intervention at youth HTS sites. Targeted quality outcomes included satisfaction with the HTS experience, intent to return for HTS within a year, and accurate knowledge of HIV prevention and transmission.

Methods

Study design

This implementation science study used time-series and pre/post analytic approaches to quantitatively test a CQI intervention [20] for AYA HTS. The time-series data included daily, individual-level, cross-sectional AYA indicators. The study period was October 2015–June 2016.

Setting

Activities were conducted in the urban setting of Nairobi, Kenya. Two clinics within Kenyatta National Hospital (KNH) were included – the Voluntary Counseling and Testing (VCT) Center and the Youth Center clinics.

Participants

All HTS HCWs aged at least 18 years were invited to participate in CQI activities and pre/post surveys. We aimed to recruit every AYA 14–24 years old who had completed HTS within the study period to participate in anonymous surveys. AYA newly diagnosed with HIV were escorted directly to the HIV care clinic following diagnosis, and were thus excluded.

Ethical considerations

The study was approved by the Kenyatta National Hospital Ethics and Research Committee (KNH ERC #P281/05/2015) and the University of Washington Institutional Review Board (UW IRB #48627). All potential participants were offered referral to the study by clinic staff; study staff assessed eligibility and consented participants. AYA at least 18 years and mature minors (14–17 years) provided oral informed consent. Adolescents aged 14–17 years who were accompanied by a caregiver provided assent, and the caregiver provided consent. A waiver of parental permission was granted for this minimal risk, anonymous survey for unaccompanied AYA. HCWs provided written informed consent.

Continuous quality improvement intervention

The intervention involved CQI training, supervision, and support to improve four HCW-identified primary indicators related to quality of AYA services through plan-do-study-act (PDSA) cycles following the Model for Improvement [21]. CQI leadership teams were formed at each site and a key driver diagram was jointly developed to determine areas for iterative, clinic-based changes. In December 2015, a 4-day in-person training introduced the Model for Improvement and its application. HCWs reviewed each clinic's baseline information for the CQI outcomes and received coaching to plan their initial PDSA cycles. Table 1 lists tested change concepts. HCWs met weekly to discuss tested changes using run charts and to plan new tests of change. Staff participated in three subsequent webinar trainings, and supervisors participated in a webinar on constructive feedback.

Table 1

Example change concepts tested.

Indicator targeted	Letter	Change concept tested	Outcome
HIV prevention and transmission knowledge	A	Clarify with HCW team that open mouth kissing is not a route of HIV transmission	Retained
	B	Emphasize methods of prevention and transmission during post-test session when information may be better absorbed than during pre-test session	Retained
	C	Poster on wall with modes of prevention and routes of transmission	Retained
	D	Try harder to remember to discuss modes of prevention and transmission	Not retained
	E	Administer pre and post counseling questionnaire about modes of prevention and transmission to AYA	Not retained
	F	Confirm and correct AYA knowledge by asking AYA to teach the information back to the HCW	Retained
Satisfaction	G	Play ‘youthful’ music at clinic; ask AYA what kinds of music they like	Retained
	H	Provide brochure about full range of services available at Youth Center, not only HTS	Retained
	I	Use all possible rooms in clinic to conduct HTS to reduce waiting time	Retained
	J	Ask AYA how satisfied they feel with services	Not retained
	K	Provide warm welcome at reception with smile	Retained
	L	Provide AYA-friendly reading material in reception area	Retained
	M	Measure waiting time at clinic	Not retained
	N	Coordination between counselors and reception to reduce waiting time for HTS	Retained
FP and STI discussion	O	Introduce FP and STI screening into counseling	Retained
	P	Incorporate FP and STI screening into standardized operating procedures (SOP)	Retained
	Q	Call referral clinic to ensure client linked to FP and STI evaluation services	Not retained
	R	3-question pregnancy prevention form placed with other HTS form as visual aid and cue	Retained
Trainings and overall changes	S	HCW training about CQI methodology
	T	Leadership training about how to interpret run charts and give feedback
	U	Use of cue cards to prompt all adopted changes
Intent to retest		No change concepts tested

AYA, adolescents and young adults; CQI, continuous quality improvement; FP, family planning; HCW, healthcare worker; HTS, HIV testing services; STI, sexually transmitted infection.

Data collection

Baseline data were collected between October and December 2015. To distinguish between baseline and intervention phases, no data were included during the second week of December. The intervention period began on the third week of December 2015. Primary outcomes, measured daily among each enrolled AYA throughout the baseline and intervention periods, were: HIV prevention knowledge, HIV transmission knowledge, satisfaction with the HTS experience, and intent to return for HTS within 1 year. HIV prevention and transmission knowledge were assessed using modified multiple-choice questions from the Demographic and Health Surveys (DHS) question guide [22]. Accurate knowledge was defined as selecting all correct options and not selecting any incorrect options. AYA satisfaction and intent to retest were measured using 5-point Likert scales throughout the baseline and intervention periods. To identify unintended negative consequences of the intervention, pre/post-HCW surveys, using 13 adapted items from the Safety Attitudes and Safety Climate Questionnaire (SAQ) short form [23], were conducted to assess HCW satisfaction as a balancing measure. Data were collected anonymously using hand-held tablets with audio computer-assisted self-interview (ACASI) and Open Data Kit (ODK) [24] for adolescent indicators and using paper forms later entered into ODK for HCW satisfaction.

Two secondary outcomes were added later in the intervention period: whether AYA reported that the HCW had discussed family planning and sexually transmitted infections (STIs) other than HIV during the HTS session. These two indicators were restricted to AYA aged at least 15 years to align with Kenyan practice guidelines. The period of October 2015 through the first week of March 2016 was considered the baseline period for these two indicators, and the third week of March was the beginning of the intervention period. All data collection ended by June 2016.

Statistical analysis

Descriptive characteristics were summarized using counts, means, proportions, medians, and interquartile ranges (IQRs). Three analytic approaches were used to assess the change associated with the CQI intervention.

First, ‘risk difference’ was estimated using interrupted time series analysis [25], which accounts for temporal trends and uses individual level data. Prais–Winsten regression (linear regression that models first order autocorrelation) was used to estimate the adjusted risk difference (aRD) associated with the CQI intervention for both continuous and binary outcomes. Secondly, to compare binary outcomes in terms of ‘relative change’ associated with CQI, generalized estimating equations with a log link, binomial family, and first-order autoregressive correlation structure were used to calculate adjusted relative risk (aRR). All models included a binary ‘step change’ term to model the immediate change associated with the intervention, a linear term to account for baseline natural temporal trends, and an interaction term to allow the intervention period temporal trend to differ from the baseline temporal trend, modeling waxing and waning of the intervention. The immediate impact of the intervention (‘step change’) is the primary focus throughout the manuscript.

Finally, CQI run chart interpretation rules were used, which define improvement in terms of shifts (6 consecutive points above or below the baseline median) and trends (5 consecutive points sequentially increasing or decreasing) [26,27]. We considered a change ‘sustained’ if it did not return to median baseline levels.

The HCW satisfaction scores were collected before and after intervention from the same HCW team; they were compared using unpaired t tests due to unlinked pre and post responses. All analyses were conducted using Stata 14 IC (StataCorp, College Station, Texas, USA). All tests were two-sided with alpha of 0.05.

Results

During the 6 weeks of baseline data collection, 172 AYA (109 from Youth Centre, 63 from VCT) were enrolled. During the 24 weeks of the CQI intervention, 702 AYA (454 from Youth Centre, 248 from VCT) were enrolled. Among all AYA who received HTS at participating clinics, 64% were referred to study staff and 89% of those referred were enrolled. The three most common reasons for nonreferral were client being in a hurry (58%), clinic staff forgetting to mention the study (17%), and clients not being interested in the study (14%). Among those referred but not enrolled, the two most common reasons were client being in a hurry (47%) and client not being interested in the study (34%).

Sociodemographic characteristics of adolescents and young adults and healthcare workers

The median age among AYA was 21 years (IQR 20–23). The majority (562/874, 64%) were enrolled in or had completed university education, and half were men. The demographic characteristics of AYA presenting at the two sites were heterogeneous in terms of age and educational status, with Youth Centre attracting a younger age demographic (37% aged 14–19 years at Youth Centre, 20% at VCT). There were 24 HCWs enrolled, representing job cadres from counselors to data clerks to drug officers, and they had a median 9 years of experience in their roles (IQR 5–11) (Table 2).

Table 2

Sociodemographic characteristics of adolescents and young adults and healthcare workers enrolled in CQI intervention at Kenyatta National Hospital, Nairobi, Kenya.

	Adolescents and Young Adults		Healthcare workersa (N = 24)
	VCT clinic (N = 311)	Youth Center (N = 563)
	n (%) or median (IQR)	n (%) or median (IQR)		n (%) or median (IQR)
Age	21 (20, 23)	21 (18, 22)	Years in current roleb	9 (5, 11)
14–19	62 (20)	207 (37)	Position
20–24	249 (80)	356 (63)	Counselor	19 (79)
Male	156 (50)	258 (46)	Data analyst	2 (8)
Highest level of education completed			Nurse	1 (4)
Primary	4 (1)	27 (5)	Program officer	1 (4)
Secondary	44 (14)	212 (38)	Drug officer	1 (4)
Polytechnic	11 (4)	14 (2)
University/college	252 (81)	310 (55)
Currently enrolled in school	273 (88)	504 (90)

CQI, continuous quality improvement; IQR, interquartile range; VCT, Voluntary Counseling and Testing.

aNot stratified by clinic.

bN = 21.

HIV and sexual and reproductive health seeking history

Most AYA (74%) presented to care specifically for HTS; however, 28% of AYA who received HTS at Youth Centre had originally presented for basic counseling services only. A majority (66%) of AYA had received HTS before. Few AYA had ever presented to clinic for family planning services (9%) or non-HIV STI testing (22%), and half (50%) were aware of where they could receive STI screening services. The majority of AYA (71%) stated intent to retest within 1 year, and AYA reported being highly likely to return to the same clinic for repeat services; a third of AYA had visited this clinic before (Table 3).

Table 3

Adolescent HIV, STI, and family planning history among adolescents and young adults seeking HIV testing at VCT clinics at Kenyatta National Hospital, Nairobi, Kenya.

	Total		VCT		Youth Center
	N	Mean, n (%); median (IQR)	n	Mean, n (%); median (IQR)	n	Mean, n (%); median (IQR)
Reason for coming to VCT	802		280		522
HIV test		591 (74)		254 (91)		337 (65)
Testing for non-HIV STIs		8 (1)		3 (1)		5 (1)
General health information or counseling		160 (20)		14 (5)		146 (28)
Family planning or pregnancy test		4 (1)				4 (1)
Other		39 (5)		9 (3)		30 (6)
First time visiting this clinic	802	510 (64)	280	186 (66)	522	324 (62)
First HIV test	802	273 (34)	280	89 (32)	522	184 (35)
Felt that healthcare worker gave sufficient information about HIV prevention	874	788 (90)	311	284 (91)	563	504 (90)
Sexual and reproductive health access (age ≥15)
Ever gone to a clinic for FP	854	78 (9)	311	22 (7)	543	56 (10)
Ever gone to a clinic for STI testing	854	186 (22)	311	67 (22)	543	119 (22)
Know of any clinic offering STI testing	854	426 (50)	311	156 (50)	543	270 (50)
HIV retesting
Timing for intended retest	874		311		563
Less than 6 months from today		489 (56)		181 (58)		308 (55)
7–12 months from today		131 (15)		50 (16)		81 (14)
More than 1 year from today		55 (6)		21 (7)		34 (6)
When I have a new partner		75 (9)		28 (9)		47 (8)
I don’t know		124 (14)		31 (10)		93 (17)
Likelihood of returning to this clinic for HIV retest (1–5)	874	4.3	311	4.3	563	4.3

FP, family planning; IQR, interquartile range; STI, sexually transmitted infection; VCT, Voluntary Counseling and Testing.

HIV knowledge

At baseline, AYA had low HIV prevention and transmission knowledge (46 and 38% with full and accurate prevention and transmission knowledge, respectively, at VCT; 48 and 18% at Youth Centre) (Table 4). The most common HIV prevention misconceptions were that HIV could not be prevented through either abstinence or through having one faithful, HIV-negative partner; among those AYA with inaccurate knowledge, 67 and 54% failed to identify these prevention techniques, respectively. The most common HIV transmission misconception was that HIV could be transmitted through ‘open mouth kissing’; among those AYA with inaccurate knowledge, 94% answered this question inaccurately.

Table 4

Impact of CQI intervention on adolescent and healthcare worker outcomes.

	Baseline		Intervention
	N	Mean, n (%)	N	Mean, n (%)		aRD	95% CI		P value	aRR	95% CI		P value
VCT
					Change associated with CQI	−0.08	−0.59	0.43	0.755
Satisfaction with testing visit (1–5)	63	4.5	248	4.5	Time trend during baseline (per week)	0.06	−0.07	0.19	0.380	–
					Change in time trend during intervention (per week)	−0.07	−0.20	0.07	0.326
					Change associated with CQI	0.30	−0.04	0.63	0.080	1.72	0.76	3.90	0.194
Correct knowledge of HIV transmission	63	24 (38)	248	178 (72)	Time trend during baseline (per week)	0.01	−0.07	0.10	0.773	1.00	0.97	1.04	0.787
					Change in time trend during intervention (per week)	−0.01	−0.10	0.07	0.763	1.00	0.96	1.03	0.781
					Change associated with CQI	0.39	0.02	0.76	0.038	2.17	0.93	5.04	0.071
Correct knowledge of HIV prevention	63	29 (46)	248	151 (61)	Time trend during baseline (per week)	−0.04	−0.14	0.06	0.411	0.99	0.96	1.02	0.396
					Change in time trend during intervention (per week)	0.03	−0.07	0.13	0.538	1.01	0.98	1.04	0.486
					Change associated with CQI	0.78	−0.11	1.67	0.087
Intent to retest within 1 year (1–5)	63	4.0	248	4.3	Time trend during baseline (per week)	−0.12	−0.36	0.11	0.299	–
					Change in time trend during intervention (per week)	0.12	−0.12	0.35	0.336
					Change associated with CQI	−0.13	−0.35	0.09	0.238	0.71	0.33	1.54	0.393
Provider discussed FP (adolescents age >15)	160	29 (18)	104	48 (46)	Time trend during baseline (per week)	0.02	0.00	0.03	0.026	1.01	1.00	1.03	0.033
					Change in time trend during intervention (per week)	0.03	0.00	0.06	0.048	1.00	0.98	1.01	0.838
					Change associated with CQI	−0.06	−0.29	0.17	0.595	0.86	0.50	1.46	0.568
Provider discussed STI (adolescents age >15)	160	61 (38)	104	57 (55)	Time trend during baseline (per week)	0.01	0.00	0.03	0.158	1.01	1.00	1.01	0.131
					Change in time trend during intervention (per week)	0.01	−0.02	0.04	0.432	1.00	0.99	1.01	0.984
Youth Center
					Change associated with CQI	0.12	−0.16	0.40	0.384
Satisfaction with testing visit (1–5)	109	4.5	454	4.5	Time trend during baseline (per week)	−0.03	−0.10	0.05	0.499	–
					Change in time trend during intervention (per week)	0.02	−0.05	0.09	0.606
					Change associated with CQI	0.42	0.21	0.63	<0.001	2.27	1.05	4.94	0.038
Correct knowledge of HIV transmission	109	20 (18)	454	289 (63)	Time trend during baseline (per week)	0.03	−0.02	0.09	0.221	1.03	0.99	1.07	0.148
					Change in time trend during intervention (per week)	−0.04	−0.10	0.01	0.147	0.97	0.93	1.01	0.127
					Change associated with CQI	−0.03	−0.26	0.19	0.759	0.95	0.59	1.53	0.826
Correct knowledge of HIV prevention	109	52 (48)	454	242 (53)	Time trend during baseline (per week)	0.00	−0.06	0.06	0.931	1.00	0.98	1.02	0.933
					Change in time trend during intervention (per week)	0.01	−0.05	0.07	0.731	1.00	0.98	1.02	0.767
					Change associated with CQI	0.39	−0.19	0.96	0.185
Intent to retest within 1 year (1–5)	109	4.0	454	4.2	Time trend during baseline (per week)	−0.06	−0.21	0.09	0.446	–
					Change in time trend during intervention (per week)	0.06	−0.10	0.21	0.460
					Change associated with CQI	0.11	−0.06	0.28	0.188	1.95	0.99	3.83	0.053
Provider discussed FP (adolescents age >15)	322	53 (16)	158	77 (49)	Time trend during baseline (per week)	0.00	−0.01	0.01	0.669	1.00	0.99	1.01	0.803
					Change in time trend during intervention (per week)	0.03	0.01	0.05	0.011	1.01	1.00	1.02	0.091
					Change associated with CQI	0.00	−0.18	0.18	0.994	1.18	0.84	1.65	0.330
Provider discussed STI (adolescents age >15)	322	145 (45)	158	110 (70)	Time trend during baseline (per week)	0.01	−0.01	0.02	0.308	1.00	1.00	1.00	0.908
					Change in time trend during intervention (per week)	0.03	0.00	0.05	0.025	1.00	1.00	1.01	0.128
Healthcare worker outcomes at VCT and Youth Center combineda
Overall satisfaction with job environment (1–5)	22	4.2	22	4.2	Change associated with CQIa	−0.04	−0.28	0.20	0.746	–	–	–	–

aRD, adjusted risk difference; aRR, adjusted relative risk; CQI, continuous quality improvement; FP, family planning; STI, sexually transmitted infection; VCT, Voluntary Counseling and Testing.

aUnpaired t test, no temporal trend adjustment made.

Bold values in the table indicate P-values <0.05 (statistically significant values). Bold italics values in the table indicate P-values >0.05 and <0.10 (trend towards significance).

Impact of continuous quality improvement on adolescents and young adults’ HIV knowledge

Controlling for temporal trends, CQI was associated with an immediate increase in accurate HIV prevention knowledge at VCT, but not Youth Centre, and increases in accurate HIV transmission knowledge at Youth Centre and a trend towards an increase at VCT. At VCT, accurate HIV prevention knowledge increased from 46% during baseline to 61% during the intervention [aRD 39 percentage points, 95% confidence interval (CI) 2 to 76, P = 0.038; aRR 2.17, 95% CI 0.93 to 5.04, P = 0.071]. At Youth Centre, the change in HIV prevention knowledge was not significant (P = 0.759). At Youth Centre, accurate HIV transmission knowledge increased from 18 to 63% (aRD 42 percentage points, 95% CI 21 to 63, P < 0.001; aRR 2.27, 95% CI 1.05 to 4.94, P = 0.038); at VCT, accurate transmission knowledge trended towards an increase from 38 to 72% (aRD 30 percentage points, 95% CI −4 to 63, P = 0.080; aRR 1.72, 95% CI 0.76 to 3.90, P = 0.194) (Table 4).

Satisfaction and intent to retest

At Youth Centre, intent to return for HTS within a year was high and did not change significantly between baseline and intervention (4.0 vs. 4.2, P = 0.185). At VCT, intent to retest trended towards an increase (4.0 vs. 4.3, P = 0.087). In both Youth Centre and VCT, satisfaction with testing was high during both baseline and intervention periods, and there was no change over time (Youth Centre: 4.5 vs. 4.5, P = 0.384; VCT: 4.5 vs. 4.5, P = 0.755) (Table 4).

Reported discussion of family planning and sexually transmitted infection services

The proportion of HCW that AYA reported having discussed family planning services increased at both Youth Centre and VCT (Youth Centre: from 16 to 49%; VCT: from 18 to 46%). Adjusted differences trended towards significance in Youth Centre in relative change, but not absolute change (aRD 11 percentage points, 95% CI −6 to 28, P = 0.188; aRR 1.95, 95% CI 0.99 to 3.83, P = 0.053), but not at VCT (P = 0.238). Proportions of HCW that AYA reported having discussed STI testing increased at both Youth Centre and VCT (Youth Centre: from 45 to 70%; VCT: from 38 to 55%), but adjusted differences were insignificant at both clinics (Table 4).

Changes in effect of intervention over time

The magnitude of the change associated with CQI grew significantly over time for three indicators: reported discussion of family planning services at both VCT and Youth Centre and reported discussion of STI at Youth Centre. For all other indicators, the change associated with CQI did not increase or decrease significantly over time (Table 4).

Run chart findings

The CQI run chart interpretation rules [26,27] had similar conclusions to interrupted time-series analyses, and also showed substantial variability in the indicators throughout the intervention period (Fig. 1a–d). There was a substantial and sustained shift in HIV transmission knowledge at Youth Centre with a 24-point shift sustained through the end of the intervention period (Fig. 1a). At VCT, there was an 8-point shift in HIV prevention knowledge, which was not sustained (Fig. 1b). At Youth Centre, there was a 6-point shift in HIV prevention knowledge midway through the intervention period that was not sustained (data not shown). There were no shifts or trends in either satisfaction (Fig. 1c) or intent to retest at either site (data not shown). There was a 12-point shift in adolescents who reported that HCW at Youth Centre discussed family planning that was sustained through the end of the intervention period (Fig. 1d), and there was a 9-point shift in adolescents who reported HCW at Youth Centre discussed STIs that was sustained through the end of the intervention (data not shown). Other run chart findings of significance, but not shown, include the following: a substantial and sustained 22-point shift in HIV transmission knowledge at VCT; a 7-point shift in reported HCW discussion of family planning at VCT that was sustained through the end of the intervention aside from one astronomical data point; a 5-point trend embedded in a 9-point shift in the reported HCW discussion of STIs at VCT, which was sustained to the end of the intervention.

Fig. 1

Selected illustrative run charts.

Panel a: CQI initially improves HIV transmission knowledge indicator, but initial success is followed by some regression to lower levels. Renewed effort with a more reliable change concept results in substantial and sustained increase in indicator. Panel b: Baseline performance is erratic and unreliable; CQI process has early success in increasing performance in HIV prevention knowledge, but indicator returns to baseline levels over time. Panel c: CQI has no impact on adolescent satisfaction, which began and remained high throughout the baseline and intervention periods. Panel d: Discussion of family planning services is low during baseline; CQI process improves this secondary outcome. Letters A through U refer to change concepts and trainings described in Table 1.

Healthcare worker satisfaction with their job environment

Healthcare workers rated their overall satisfaction with their jobs as 4.2/5 at baseline and 4.2/5 during the intervention (P = 0.746). None of the 13 components of job satisfaction changed substantially or significantly (P > 0.1 for all comparisons).

Discussion

The DASH implementation science study found that CQI at youth HTS clinics in Kenya was associated with substantially improved AYA HIV prevention and transmission knowledge, and a trend towards improved intent to retest at one clinic. Additionally, AYA satisfaction with the HTS visit experience and intent to retest both began and remained high throughout the study period. CQI did not negatively impact HCW satisfaction.

As Joint United Nations Programme on HIV/AIDS (UNAIDS) and President's Emergency Plan for AIDS Relief (PEPFAR) prioritize reaching the 90-90-90 goals for AYA and reducing AYA HIV incidence [28], it is increasingly important to identify innovative approaches to improve the quality of AYA HIV services. Gaps in achieving AYA HIV goals are not due to absence of HIV testing and treatment commodities or providers, but rather in the quality of services offered [1]. However, improving service quality, as opposed to increasing service quantity, can be challenging; perceptions of quality are difficult to measure and have many underlying influencing factors. Historically, there have been few interventions that have resulted in robust improvements in AYA health service quality in resource-limited settings [29]. However, this study joins a growing, but mixed, body of literature that describes interventions – primarily large, national-level policy changes – that improve the quality of AYA services [30-33].

This is the first study to our knowledge to systematically evaluate CQI in an adolescent HTS setting aimed at improving quality of services. This study is unique in that it was able to observe changes in adolescent understanding of HIV prevention and transmission information, and also intent to retest – indicators that are closer to health impact than HCW-controlled process indicators. Additionally, CQI is a generalizable process that allows for locally informed changes to address unique issues at the clinic level. This intervention has potential for scalability to address AYA HIV prevention and management goals.

The study observed low levels of AYA HIV knowledge during baseline. AYA HIV knowledge gaps persist worldwide and may contribute to high HIV incidence. The gaps observed in the DASH study differed somewhat from the results from the Kenyan DHS surveys; condom knowledge, mother-to-child transmission knowledge, and mosquito transmission knowledge were higher among our study population than among age and sex-specific levels nationally, whereas abstinence and having one monogamous, uninfected partner knowledge were lower than national averages [34]. This may be due to slightly different wording of questions (e.g. ‘prevent HIV’ versus ‘reduce risk of HIV’) and high levels of education among our urban population of AYA. In settings where access to comprehensive sexual education is not widely available, the HTS visit may serve a critical role in providing accurate HIV knowledge for AYA. This study also observed low levels of historic adolescent engagement with family planning and STI services, even among a population of AYA who self-selected to present for HTS. HTS may be an important entry point for sexually active AYA to access comprehensive sexual and reproductive health services.

After large and sustained improvement had been observed in the original HIV knowledge indicators, family planning and STI indicators were added as secondary outcomes; it is common in CQI that once initial goals are achieved, new goals are addressed. However, conclusions about the impact of CQI on the family planning and STI indicators were mixed. Crude differences in proportions and traditional run chart interpretation rules all suggested a large intervention impact on these two indicators. However, analyses that modeled the temporal trends during the baseline and intervention periods did not suggest an immediate impact of the intervention, but rather noted that the intervention improved these indicators over time.

The success of the CQI intervention in this clinic setting may have been due to a well primed environment for change. The AYA focus of this intervention was directly informed by previous qualitative work with this clinical team in which HCW noted that they felt ill prepared to address the unique and challenging health needs of AYA seeking HTS [8]. Team members were directly involved in developing the CQI intervention and in determining the indicators targeted for change. In addition to a highly motivated clinical team, this intervention also benefitted from strong leadership buy-in and direct involvement in supervision of the CQI process. The main challenge faced during the intervention was maintaining morale and momentum for change; others have observed that routinely identifying errors to change service delivery is challenging when there is judgment attached to poor HCW performance [35]. Substantial effort was made to frame the tone of weekly data reviews with HCW to focus on achievements, rather than absence of change.

The study benefitted from a large sample of AYA seeking HTS in a public hospital, high levels of data completeness, and robust analytic methods. To limit social desirability and recall bias, AYA entered responses using ACASI and completed surveys immediately after the HTS session. There were limitations, however, in certain aspects of design, population, and analysis. There remained a high level of variability in several indicators, and there were AYA who were not approached and not referred to the study. It is possible that there was differential referral, though it would not be expected to vary over time and therefore would not be expected to affect the relative change observed. Our sample is limited in generalizability, as the majority were educated, HIV-negative, young adults attending a tertiary facility, who may not have the same experiences as younger or HIV-infected adolescents in other settings. A pre-post design is not as robust as a design that employs a concurrent control group; however, interrupted time-series analysis is a rigorous analytic method that accounts for both autocorrelation and temporal trends and generally results in more valid estimates than pre-post analysis [25]. Additionally, this analytic technique tended to have similar conclusions to CQI analytic methods for run chart interpretation, increasing the strength of study findings.

Continuous quality improvement offers a team-driven approach for improving quality of care. The staff ownership of the improvement process and adaptability of the intervention to address future quality outcomes were considered strengths. However, our intervention included dedicated staff to assist with data collection and analysis, which would be difficult in typical public health facilities in resource-limited settings. CQI has limitations in complexity and cost, and may be comparably effective to training interventions for targeted change such as knowledge improvement. Additional studies are needed to determine in which environments this intervention is most effective, what kinds of pre-CQI activities are needed to prime a site to be ready for change, and what the long-term sustainability of CQI is in resource-limited settings in the absence of additional resources to support the CQI process.

Conclusion

In summary, this CQI intervention was associated with substantial, significant, and sustained changes in AYA knowledge of HIV prevention and transmission. AYA satisfaction and intent to retest for HIV within a year began and remained high throughout the intervention period, and CQI did not have a detrimental effect on HCW satisfaction. CQI may be a valuable intervention for effecting locally-informed, flexible change in the quality of HIV service delivery for AYA in SSA.

Acknowledgements

Author contributions: A.D.W., C.M., S.B.M., P.M.M., D.C.W., D.B., I.N.N., G.O., G.J.S., J.A.S., and P.K.K. conceptualized the study design. A.D.W., C.M., S.B.M., P.M.M., D.C.W., D.B., J.N., I.N.N., G.O., J.A.S., and P.K.K. led development of the study materials. C.M., D.C.W., J.N., and A.D.W. oversaw data collection. A.D.W. conducted the analysis with input from S.B.M., J.N., G.J.S. J.A.S., and P.K.K. A.D.W. drafted the manuscript and all authors contributed to the final manuscript.

The authors thank the DASH study participants and their families, without whom this research would not be possible. The authors also thank the Kenyatta National Hospital staff at the VCT, Youth Center, and tents for their tremendous effort in the CQI process and in recruiting study participants. The authors thank the DASH administrative and data team for their dedication and support; and the Kenyan National AIDS and STI Control Programme (NASCOP) for valuable input during study design, conduct, and dissemination. Lastly, the authors thank members of the Kizazi Working Group (UW Global Center for Integrated Health of Women, Adolescents and Children (Global WACh)) and Kenya Research & Training Center (KRTC) for their support during the preparation of this article.

Research reported in this publication was supported by NIAID, NCI, NIMH, NIDA, NICHD, NHLBI, NIA, NIGMS, NIDDK of the National Institutes of Health under award number AI027757 and by a supplement to this award. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. Funding sources were not involved in the analyses or interpretation of data.

Conflicts of interest

The authors have no conflicts of interest.