Increased reengagement of out-of-care HIV patients using Lost & Found, a clinic-based intervention.

BACKGROUND: Negative health outcomes associated with being out of HIV care (OOC) warrant reengagement strategies. We aimed to assess effectiveness of Lost & Found, a clinic-based intervention to identify and reengage OOC patients.
METHODS: Developed and delivered using implementation science, Lost & Found consists of two core elements: identification, operationalized through nurse validation of a real-time list of possible OOC patients; and contact, via nurse-led phone calls. It was implemented over a 12-month period (2018-2019) at the Chronic Viral Illness Service, McGill University Health Centre (CVIS-MUHC) during a type-II implementation-effectiveness hybrid pilot study. Descriptive outcomes of interest were identification as possibly OOC, OOC confirmation, contact, and successful reengagement. We present results from a pre-post analysis comparing overall reengagement to the year prior, using robust Poisson regression controlled for sex, age, and Canadian birth. Time to reengagement is reported using a Cox proportional hazards model.
RESULTS: Over half (56%; 1312 of 2354) of CVIS-MUHC patients were identified as possibly OOC. Among these, 44% (n = 578) were followed elsewhere, 19% (n = 249) engaged in care, 3% (n = 33) deceased, 2% (n = 29) otherwise not followed, and 32% (n = 423) OOC. Of OOC patients contacted (85%; 359/423), 250 (70%) reengaged and 40 (11%) had upcoming appointments; the remainder were unreachable, declined care, or missed given appointments. Pre-post results indicate people who received Lost & Found were 1.18 [95% confidence interval (CI) 1.02-1.36] times more likely to reengage, and reengaged a median 55 days (95% CI 14-98) sooner.
CONCLUSION: Lost & Found may be a viable clinic-based reengagement intervention for OOC patients. More robust evaluations are needed.

Introduction

The UNAIDS Fast-Track targets to end the HIV epidemic by 2030 prioritized reengagement of people with HIV (PWH) who are out of care (OOC) [1]. Specific definitions for ‘OOC’ or ‘loss-to-follow-up’ vary but all include some criteria related to gaps in HIV care [2-6]. OOC patients are at risk for individual-level negative health outcomes leading to broader population-level health and economic impacts. These include suboptimal combination antiretroviral treatment (cART) adherence [7,8], viremia [8-10], development of HIV resistance [11,12], opportunistic infections [13], hospitalizations [14], mortality [15-17], and secondary HIV transmission [9,18].

Data-to-Care (D2C) is ‘a public health strategy that uses HIV surveillance data and other data sources’ to identify people with gaps in HIV testing or care, subsequently directing them for reengagement by public-health officials [5,6,19-27]. In combination with services to locate, contact, and facilitate access to HIV medical care, D2C has been shown to be an effective strategy [24] and is recommended by the US Centers for Disease Control and Prevention (CDC) for improving the HIV continuum of care [19,20]

Surveillance-based D2C strategies, however, are limited in their capacity to accurately identify OOC patients [3,28-31] and, as such, often require supplemental input from clinicians [25,30]. The large, multijurisdictional datasets or data-sharing agreements used in surveillance-based D2C [20,24,26,32-34] may also be infeasible in some jurisdictions because of legal barriers or poor public-health infrastructure [35,36], further limiting the utility of these strategies in some contexts.

Two studies utilizing clinic-based approaches for identification and reengagement of OOC patients successfully reengaged approximately half of OOC patients [4,37]. Although these clinic-based interventions show promise, evaluations of effectiveness were limited. Most studies of reengagement interventions have reported descriptive results only and none have been guided by implementation or sustainability frameworks [3-6,21-27,29-33,37-41]. This lack of theoretical framing may limit their long-term uptake and sustainability [42].

Lost & Found is a clinic-based intervention developed and delivered using implementation science, informed by previous efforts to identify and reengage OOC patients [4-6,37,41,43]. In contrast to traditional D2C, Lost & Found leverages available clinical datasets and related clinical knowledge to improve identification and reengagement of OOC patients.

This study aims to determine effectiveness of Lost & Found as a reengagement intervention. Specifically, we conducted a descriptive analysis of reengagement outcomes and a pre-post evaluation of total reengagement and time to reengagement among OOC patients.

Methods

Results are drawn from a type II implementation-effectiveness hybrid pilot study of Lost & Found [44]. A full study protocol is published elsewhere [43] and selected details are discussed here. This study was approved by the McGill University Health Centre (MUHC) research ethics board (2018-4369).

Setting

The MUHC is a large public quaternary care hospital in Montréal, Canada. Multidisciplinary care for adult patients with chronic viral illnesses, such as HIV, hepatitis B, and hepatitis C, is provided at the Chronic Viral Illness Service (CVIS-MUHC) clinic. Over 90% of patients who received care at the CVIS-MUHC in 2018 were PWH (n = 1777). Before Lost & Found, approximately 10% of patients did not return for care annually, and no formal system to identify or reengage patients existed [43]. Ad hoc and periodic efforts by clinical nurses to identify and reengage OOC patients, using patient lists, were limited by several barriers including competing work priorities, staffing shortages, technology limitations, and lack of decision support to accurately identify OOC patients [43].

Lost & found intervention

Lost & Found is an intervention to identify and reengage OOC patients into HIV care. It was implemented at the CVIS-MUHC between April 2018 and 2019 to circumvent barriers to OOC patient reengagement and reduce HIV care attrition. It consists of two core, evidence-based elements adapted to the clinic: identification and documentation of OOC patients; and systematic contact of OOC patients.

For the first core element – identification and documentation of OOC patients – a list of OOC patients and an OOC risk prediction tool (OOC-RPT; Fig. 1) were used. Both were integrated into the clinic's electronic medical record (EMR) database. An automated portion of the OOC-RPT (Fig. 1, step 1: triage) used available clinical information to categorize patients as high, intermediate, or low risk of HIV disease progression [43]. On the basis of time since last appointment and risk category, patients were classified as engaged in care or possibly OOC. Possible OOC patients were placed on a real-time OOC list, visible upon launching the EMR application (Supplemental Digital Content 1). Using this information, nurses then accessed patients’ clinical files to validate OOC status, either retaining or removing them from the list (Fig. 1, step 2: nurse validation). Step 1 of the OOC-RPT was designed to capture any patient potentially requiring reengagement by CVIS-MUHC nurses. Consequently, it is highly inclusive, reflecting nurses’ concerns about potentially excluding at-risk patients. These possible misclassifications of OOC are corrected by step 2 of the OOC-RPT, when nurses validate OOC status and retain only those patients in need of reengagement.

Fig. 1

Out of care risk prediction tool.

In the second core element, confirmed OOC patients were contacted. This consisted of prioritization of higher risk patients, nurse-delivered phone calls, multiple contact attempts (if necessary), and motivational communication. The OOC list was arranged to hierarchically organize patients, with high-risk patients preceding intermediate-risk and low-risk patients, as a means of facilitating prioritization of phone calls. The OOC list was managed by nurses who attempted to contact patients at regular intervals until they were reengaged or it was learned that care was being received elsewhere. Recommended timelines for contacting and reengaging patients were co-developed with nurses but they were encouraged to use clinical judgement in determining the urgency with which individual patients should be reengaged [43].

Participants and analyses

Objective I: descriptive analysis

All CVIS-MUHC patients in the EMR database during the 12-month implementation phase were included. The primary outcomes of interest were identification by the OOC-RPT, confirmation of OOC status by nurses, contact attempts, and successful reengagement. We also provide other outcomes from nurses’ OOC status validations, such as the number of patients followed elsewhere and reasons for non-reengagement. For patients marked possibly OOC and reengaged multiple times throughout the implementation phase, only reengagement attempts related to the first OOC event were analysed. We report both overall and risk-category-stratified (high, intermediate, low) intervention metrics, including number of contact attempts needed to reengage patients and time to reengagement, as well as routinely collected clinical (e.g. CD4+ cell counts, HIV viral load) and sociodemographic information.

Objective II: pre-post analyses

For the pre-post analyses, we retroactively applied the OOC-RPT to patients in the pre-implementation (15 April 2017 to 15 April 2018) and implementation (15 April 2018 to 15 April 2019) phases excluding information from the nurse validation step of the implementation phase. This modified version of the OOC-RPT, a ‘pseudo-RPT’, was used to minimize differential misclassification bias of OOC status with respect to reengagement as changes to OOC statuses through nursing input were not possible in the pre-implementation phase (i.e. the comparator) as they were in the implementation phase. Excluding this information nondifferentially increases the total number of people included in each phase, all of whom would have been marked ‘not OOC’ by nurses. Consequently, we render the groups more comparable, but with the trade-off of diluting any effect that may exist. More detailed explanations of these considerations are described in Supplemental Digital Content 2. All patients marked OOC by the pseudo-RPT in the implementation and pre-implementation phases were included in this analysis. For each patient, only the first OOC event in each phase was considered. OOC patients are right censored at the end of each phase, meaning their time since being marked OOC is terminated at reengagement or the end of the given study phase.

To determine the risk of reengagement for patients deemed OOC by the pseudo-RPT, we used a Poisson regression model with robust variance estimation and a log link function [43,45]. Robust Poisson regression for binary outcomes is a more reliable alternative to logistic regression and provides more easily interpreted results [46,47]. We modelled reengagement as a function of study phase, controlled for sex, age, and being born in Canada [43]. These variables were selected as they may be associated both with receiving the intervention and reengagement [43,48]. Other variables were thought unlikely to differ between the two phases, or were not available in the EMR system. The risk difference in reengagement between the two phases was determined using the same model but with an identity link function [45]. In addition to these preplanned analyses, we conducted a survival analysis using a Cox proportional hazards model to provide median time to reengagement. The results of this Cox model, as well as a supplemental propensity score-adjusted robust Poisson model, were also used to validate results from the preplanned analysis. Full results and analysis plans for these models are presented in Supplemental Digital Content 3.

Results

Descriptive analyses

Intervention results over the 12-month implementation phase are summarized in Fig. 2. Of the 2354 patients with HIV in the clinical EMR database, 56% (n = 1312) were marked as possibly OOC using the OOC-RPT. Of this group, nurses determined 44% (n = 578) were followed elsewhere, 32% (n = 423) were OOC, 19% (n = 249) were still engaged in care, 3% (n = 33) were deceased, and 2% (n = 29) were of unknown status. Only six patients had their status changed from engaged in care to possible OOC. Notably, of the 249 patients marked as possibly OOC but still engaged, the large majority (75%) had medical follow-up visits booked just beyond the window period of their assigned risk category, while the remainder had shared care with another clinic or other arrangements. Among patients confirmed OOC, nurses attempted to contact 85% (n = 359); the remaining 15% (n = 64) had already scheduled appointments at the time of nurse review or were unreachable until they reengaged on their own.

Fig. 2

Summary of Lost & Found outcomes: (a) among all patients in the clinical electronic medical record, (b) among out of care patients only, and (c) among out of care patients contacted but not yet reengaged.

Among the OOC patients for whom contact was attempted, 70% (n = 250) were successfully reengaged as of the end of the intervention/implementation phase. Of those not reengaged (n = 109), 46% (n = 50) were left a voice message, 30% (n = 33) had upcoming appointments beyond study end and the remaining 24% (n = 26) were unreachable, declined a reengagement visit, or had missed their reengagement visit. Two-thirds (64%; 21/33) of patients with upcoming appointments after the end of the implementation phase missed their first scheduled reengagement visit.

Baseline characteristics of contacted OOC patients (n = 359), stratified by reengagement, are presented in Table 1. The OOC-RPT, including nurse validation, classified 11% (n = 40) of patients as high risk, 73% (n = 263) intermediate risk, and 16% (n = 57) low risk.

Table 1

Baseline characteristics of contacted out of care patients, stratified by reengagement.

	OOC, contacted	Contacted and reengaged	Contacted, not yet reengaged
N	359	250	109
Sex [n (%)]
Female	124 (34.5%)	87 (34.8%)	37 (33.9%)
Male	235 (65.5%)	163 (65.2%)	72 (66.1%)
Age, median [IQR]	51 [42–57]	52 [42–58]	50 [40–56]
Born in Canada [n (%)]	155 (43.2%)	108 (43.2%)	47 (43.1%)
Years since HIV diagnosis, median [IQR]	16 [11–23]	17 [11–23]	16 [11–22]
Risk categorya
High	40 (11.1%)	26 (10.4%)	14 (12.8%)
CD4+ cell count <100	14 (3.9%)	12 (4.8%)	2 (1.8%)
CD4+ cell count 100–200 + VL ≥40	12 (3.3%)	8 (3.2%)	4 (3.7%)
New patient	3 (0.8%)	1 (0.4%)	2 (1.8%)
Other/clinical judgement	11 (3.0%)	5 (2.0%)	6 (5.5%)
Intermediate	262 (73.0%)	197 (78.8%)	65 (59.6%)
CD4+ cell count 100–300 + VL < 40	32 (8.9%)	27 (10.8%)	5 (4.6%)
CD4+ cell count ≥200 + VL ≥40	50 (13.9%)	29 (11.6%)	21 (19.3%)
Non-ART polypharmacy (>5 non-antiretrovirals)	107 (29.8%)	83 (33.2%)	24 (22.0%)
History of chronic HCV infectionb	30 (8.4%)	21 (8.4%)	9 (8.3%)
Youth (<25 years old)	13 (3.6%)	11 (4.4%)	2 (1.8%)
CD4+ cell count nadir <200	155 (43.2%)	126 (50.4%)	29 (26.6%)
Other/clinical judgement	3 (0.9%)	2 (0.8%)	1 (0.9%)
Low	57 (15.9%)	27 (10.8%)	30 (27.5%)
CD4+ cell count ≥300 + VL <40	53 (14.8%)	25 (10.0%)	28 (25.7%)
Other/clinical judgement	4 (1.1%)	2 (0.8%)	2 (1.8%)
History of chronic HCV infectionb [n (%)]	42 (11.7%)	30 (12.0%)	12 (11.0%)
CD4+ cell count nadirc (cells/μl), median [IQR]	184 [78–335]	164 [71–310]	274 [134–474]

Among contacted OOC patients at the CVIS-MUHC clinic from 15 April 2018 to 15 April 2019 (n = 359). ART, antiretroviral therapy; IQR, interquartile range; OOC, out of HIV care; VL, viral load.

Risk category criteria, by risk category: High: CD4+ cell count less than 100 cells/μl (irrespective of viral load) or CD4+ cell count 100–200 cells/μl with viral load greater than 40 copies/ml or New patient. Intermediate: CD4+ cell count 100–300 cells/μl with viral load less than 40 copies/ml or CD4+ cell count greater than 200 cells/μl + viral load greater than 40 copies/ml or non-ART polypharmacy (>5 non-antiretrovirals) or Hx of chronic HCV infection (HCV RNA+) or Youth (<25 years old) or CD4+ cell count nadir less than 200 cells/μl. Low: CD4+ cell count greater than 300 cells/μl + viral load less than 40 copies/ml. Risk categories are mutually exclusive. For the criteria within each risk category, only the CD4+ cell count and viral load criteria are mutually exclusive from each other; otherwise multiple criteria can apply to the same patient. ‘Other’ refers to a nurse defined reason for classification into the given risk category.

Ever HCV RNA+.

Lowest CD4+ cell count on record before reengagement.

Clinical characteristics and intervention metrics for reengaged OOC patients are presented in Table 2. CD4+ cell counts and viral loads reflected the criteria of the OOC-RTP. Both at reengagement and last visit, patients in the low-risk category had the highest CD4+ cell counts and lowest viral loads, while patients in the high-risk category had the lowest CD4+ cell counts and highest viral loads. The occurrence of viremia (200+ copies/ml) increased from 12% (n = 30) at last visit to 15% (n = 38) at reengagement, despite 79% (n = 197) having undetectable viral loads (<40 copies/ml) at each time point.

Table 2

Characteristics of patients contacted and reengaged.

Risk category		Overall	High riska	Intermediate riska	Low riska
N (%)		250	26 (10%)	197 (79%)	27 (11%)
CD4+ cell count
CD4+ cell count; median [IQR]	Reengagement	528 [351–744]	151 [72–314]	544 [379–770]	704 [574–850]
	Previous visit	538 [350–731]	139 [84–257]	551 [378–730]	751 [612–873]
CD4%, median [IQR]	Reengagement	30 [21–38]	12 [5–17]	31 [23–38]	35 [30–41]
	Previous visit	28 [21–37]	12 [7–14]	29 [23–37]	34 [30–38]
CD4+/CD8+ ratio, median [IQR]	Reengagement	0.70 [0.40–1.10]	0.20 [0.10–0.30]	0.70 [0.50–1.10]	1.10 [0.75–1.35]
	Previous visit	0.70 [0.40–1.08]	0.20 [0.10–0.30]	0.70 [0.50–1.07]	1.00 [0.75–1.20]
VLb
VL, median [IQR]	Reengagement	<40 [<40 to <40]	116 [<40 to 37 646]	<40 [<40 to <40]	<40 [<40 to <40]
	Previous visit	<40 [<40 to <40]	203 [<40 to 21 376]	<40 [<40 to <40]	<40 [<40 to <40]
Undetectable VL [n (%)]	Reengagement	197 (79%)	11 (42%)	162 (82%)	24 (89%)
	Previous visit	197 (79%)	7 (27%)	163 (83%)	27 (100%)
Viremia [n (%)]	Reengagement	38 (15.2%)	12 (46.2%)	24 (12.2%)	2 (7.4%)
	Previous visit	30 (12.0%)	13 (50.0%)	17 (8.6%)	0 (0.0%)
Days from… (median [IQR])	…Previous visit to reengagement	315 [246–448]	229 [168–428]	304 [247–422]	444 [427–514]
	…First contact attempt to reengagement	72 [34–136]	86 [37–107]	78 [34–145]	57 [34–78]
Any missed visits reengagement visits [n (%)]		92 (37%)	14 (54%)	73 (37%)	5 (19%)
Number of contact attempts, median [IQR]		2.0 [1.0–4.0]	4.0 [1.5–5.0]	2.0 [1.0–4.0]	2.0 [1.0–2.5]

Among OOC patients contacted and reengaged into care at the CVIS-MUHC clinic from 15 April 2018 to 15 April 2019 (n = 250). IQR, interquartile range; OOC, out of HIV care; VL, viral load.

Risk category criteria: High: CD4+ cell count less than 100 cells/μl (irrespective of viral load) or CD4+ cell count 100–200 cells/μl with viral load greater than 40 copies/ml or new patient. Intermediate: CD4+ cell count 100–300 cells/μl + viral load less than 40 copies/ml or CD4+ cell count greater than 200 cells/μl with viral load greater than 40 copies/ml or non-ART polypharmacy (>5 non-antiretrovirals) or Hx of chronic HCV infection (HCV RNA+) or youth (<25 years old) or CD4+ cell count nadir less than 200 cells/μl. Low: CD4+ cell count greater than 300 cells/μl with viral load less than 40 copies/ml

Undetectable = viral load under 40 copies/ml; viremia = viral load over 200 copies/ml.

The median time from previous visit to reengagement was 315 days (IQR 246–448), with 72 days (IQR 34–136) between first contact attempt and reengagement. Nurses made a median of two contact attempts (IQR 1.0–4.0) for each patient, and over a third of patients (37%; n = 92) missed at least one scheduled reengagement visit before actually being reengaged. High-risk patients took the longest to reengage and received the most contact attempts, requiring 86 days (IQR 37–107) and four contact attempts (IQR 1.5–5.0) compared with 78 days (IQR 34–145) and two attempts (IQR 1.0–4.0) for intermediate-risk patients, or 57 days (IQR 34–78) and two attempts (IQR 1.0–2.5) for low-risk patients. The proportion of patients who missed a scheduled reengagement visit also increased by risk category.

Pre-post analyses

Results are presented in Table 3. A total of 740 and 804 patients were marked OOC by the pseudo-RPT in the pre-implementation and implementation phases, respectively.

Table 3

Results from the Poisson regression model with robust variance estimation, overall and by risk category.

Variableb	Estimatea (95% CI)	P value
All risk categories (implementation phase: n = 804; pre-implementation phase: n = 740)
(Intercept)	0.576 (0.500–0.664)	<0.001
Imp	1.179 (1.022–1.359)	0.023
Sex	0.918 (0.783–1.076)	0.290
Age	0.999 (0.993–1.005)	0.795
Canada	0.799 (0.686–0.931)	0.004
High-risk category (Implementation phase: n = 199; pre-implementation phase: n = 128)
(Intercept)	0.665 (0.487–0.909)	0.011
Imp	1.201 (0.879–1.640)	0.250
Sex	0.835 (0.613–1.139)	0.256
Age	1.005 (0.994–1.016)	0.391
Canada	0.762 (0.533–1.090)	0.137
Intermediate-risk category (implementation phase: n = 481; pre-implementation phase: n = 493)
(Intercept)	0.601 (0.505–0.716)	<0.001
Imp	1.116 (0.938–1.329)	0.216
Sex	0.905 (0.742–1.104)	0.327
Age	0.999 (0.992–1.007)	0.872
Canada	0.889 (0.740–1.067)	0.207
Low-risk category (implementation phase: n = 124; pre-implementation phase: n = 119)
(Intercept)	0.345 (0.222–0.536)	<0.001
Imp	1.658 (1.029–2.671)	0.038
Sex	1.018 (0.606–1.710)	0.947
Age	0.986 (0.965–1.007)	0.184
Canada	0.536 (0.326–0.881)	0.014

CI, confidence interval.

Risk for ’(Intercept)’, risk ratio otherwise.

‘Imp’ is a binary variable for being marked out of HIV care (OOC) in the implementation phase, compared with the pre-implementation phase.

‘Sex’ is a binary variable for the patient's sex, as documented in their electronic medical record, where females are assigned a value of 1.

‘Age’ is the patients age, centered at age 50 years.

‘Canada’ is a binary variable where patients born in Canada are assigned a value of 1.

After 1 year of the Lost & Found intervention, patients identified as OOC were 1.18 (95% CI 1.02–1.36) times as likely to be reengaged compared with those in the pre-implementation phase after controlling for sex, age, and being born in Canada. This is equivalent to reengaging 72 (95% CI 8.97–134.45) OOC patients who would not otherwise have been reengaged. On the basis of results from the Cox proportional hazards model adjusted for the same covariates, median time to reengagement for OOC patients receiving Lost & Found in the implementation phase was 162 days (95% CI 147–183) compared with 217 days (95% CI 197–245) in the pre-implementation phase.

When stratified by risk category, Lost & Found primarily impacted reengagement in the low-risk category, where OOC patients in the implementation phase were 1.66 (95% CI 1.03–2.67) times more likely to be reengaged compared with those in the pre-implementation phase. High-risk and intermediate-risk patients who received Lost & Found were no more likely to reengage than those in the pre-implementation phase.

The results from the propensity score-adjusted robust Poisson model and Cox proportional hazards model mirror these findings. Full results of propensity score and survival analyses, including outputs from the Cox model, adjusted cumulative incidence curves, as well as tests of the model's assumptions, are presented in Supplemental Digital Content 3.

Discussion

Our study demonstrates that Lost & Found may be an effective alternative to centralized, D2C approaches in the identification and reengagement of OOC patients. Through the OOC-RPT, nurses were highly successful in validating the HIV care status of most patients identified as possibly OOC. Only 2% (n = 29) were of unknown care status, compared with 14–35% in other interventions [5,6,21,25-27]. Moreover, over a quarter of our original patient pool (n = 641; 27%) were identified as no longer active patients, and the majority were followed elsewhere (n = 578). This reduced the active care list during the first few months of the study, effectively focusing nurses’ reengagement efforts. Integration of the nurse validation step of the intervention appears to be important in the identification of OOC patients for whom reengagement efforts are needed. Compared with surveillance-based D2C approaches, where at least 43% are falsely marked OOC, Lost & Found appears more accurate [3,26,29-31,49]. This initial clean up and accuracy in identifying OOC patients may address some of the high effort and low yield concerns often attributed to reengagement efforts [50].

A large majority (70%) of OOC patients contacted during the implementation phase were successfully reengaged before study end. Although variability in OOC definitions and reporting with respect to patient disposition may limit the capacity for between-study comparisons, this proportion of reengaged patients is similar to others, ranging from 44 to 94% [4-6,22,26,27,37]. Many patients required several contact attempts and among the 33 patients with upcoming appointments at the end of the implementation phase, 61% missed their first scheduled visit. Other studies of interventions to identify and reengage OOC patients have not evaluated phone calls separately from a full intervention package, and as such provide little guidance about the number of contact attempts needed to reengage OOC patients [4-6,21,22,24-27]. Our results, along with a higher level of viremia at reengagement, highlight the importance of ongoing and timely reengagement in this population.

The effort required to reengage patients appears to increase by risk category. Despite being targeted for more rapid reengagement, high-risk patients required more time to reengage and more contact attempts [43]. Over half of high-risk patients (n = 14; 54%) missed at least one scheduled reengagement visit. Social determinants and related barriers (e.g. income, transportation, work or child-care conflicts) are likely major contributors to reengagement difficulties [51]. Information on sociodemographics and reasons for becoming OOC was collected as part of this study; these data will be analyzed and summarized in a forthcoming publication.

Results from our pre-post analyses indicate that OOC patients are more likely to reengage and to do so earlier in the context of Lost & Found. Although positive, these results represent a very conservative evaluation of the intervention's effectiveness. By using a pseudo-RPT in these analyses (i.e. an artificial version of the OOC-RPT where nurse validation is not included), we drastically increased inclusivity in how OOC patients were identified. This added patients in each phase who would normally have been removed from the OOC list by nurses, and likely biased our results to the null. This increase is demonstrated when comparing the number of patients captured by the pseudo-RPT in the implementation phase of the pre-post analysis (n = 804) to the number confirmed OOC of the descriptive analysis (n = 423), where the true OOC-RPT was used including nurse validation. A large difference existed for all risk categories but the largest proportional difference occurred in the high-risk category where only 26% (52/199) of those marked by the pseudo-RPT were truly OOC, indicating a high level of misclassification. The null bias is most evident in the high-risk and intermediate-risk groups of our stratified analyses. Despite nondifferential bias across all categories, we still observe an effect overall as well as in the low-risk category. The intervention is likely effective for other risk categories but requires further evaluation with more appropriate study designs.

This study has several limitations. First, while all pre-post designs are vulnerable to uncontrolled time-related confounding, to our knowledge, there were no major changes in clinical practice guidelines or how patients were followed over the period of analysis. Second, these analyses are limited to only the first OOC event in each period, primarily because of censoring imposed by the pre-post design and to ensure comparability between phases. As a result, we provide no information about patient-level effectiveness of Lost & Found after first successful reengagement or its impact on OOC rates over the long-term. There were 40 patients marked OOC more than once in the implementation phase, all but one of whom were reengaged again before the end of the study period. Third, our robust Poisson model does not account for censoring or provide information about timeliness of reengagement. These are resolved by our secondary analysis using a Cox proportional hazards model. Despite accounting for censoring, the Cox proportional model provided similar results to the robust Poisson model, with the exception that a significant effect of the intervention was observed in the intermediate-risk category of the stratified analysis (Supplemental Digital Content 3). Nonetheless, we only reported details of the robust Poisson analysis for consistency with our protocol and to simplify understanding of our results, given the challenges inherent in hazard ratio interpretation [52,53]. Finally, this study provides little information about effectiveness of the first core element, identifying OOC patients. The intermediate-risk category of the automated portion of our OOC-RPT is likely overly sensitive, largely because of the CD4+ cell counts nadir criterion. However, at the behest of nurses, the primary stakeholders in Lost & Found's implementation, OOC-RPT criteria were left unchanged. Their prioritization of patients with lower CD4+ cell counts nadirs is likely reflected in the higher median pre-reengagement CD4+ cell counts nadirs of contacted OOC patients not yet reengaged (median: 274 cells/μl; IQR: 134–474) compared with OOC patients successfully reengaged (164 cells/μl; IQR: 71–310). We depended on nurses’ clinical judgement to improve accuracy of the OOC-RPT but additional strategies could be used. For example, our OOC-RPT does not consider longitudinal information. Patient trajectories with regard to OOC could be important in determining, which patients will become OOC again [54].

Lost & Found may be a viable, decentralized alternative to D2C approaches to identify and reengage PWH who are OOC. This clinic-based intervention may be especially pertinent in settings with limited capacity to undertake public-health or surveillance-based D2C strategies. Most reengagement interventions depend upon ongoing human resources to supplement existing clinical care, including work by public-health employees or creating new roles in the clinical team [21-25,27,37]. Lost & Found was integrated into existing clinical practice and infrastructure; this meant early investment in EMR improvements and support from an internal facilitator to aid in implementation [43]. Costing is beyond the scope of this study but the limited human resource requirements for this approach may offset early technology-related costs for changes to the clinical EMR. As this intervention requires minimal ongoing financial support and has buy-in from primary implementation stakeholders, sustainability over the long-term is more likely. Consistent with the adaptation inherent to an implementation science approach, clinics with limited technological capacity might consider scaling back automated aspects of the OOC-RPT and creating simpler OOC definitions. Such definitions might prioritize missed visits, which are easily monitored and appear to be strong predictors of future OOC, HIV disease-related health outcomes as well as mortality [17,55-58]. However, this might come at the expense of targeted reengagement for higher risk groups.

Although promising, more robust evaluations of Lost & Found are needed, particularly given the many adjustments and assumptions to accommodate the study design and inadequate control group. A forthcoming mixed-methods analysis of implementation data will provide additional insights, specifically regarding the utility of implementation strategies as well as other factors in the delivery and sustainability of the intervention. Knowledge of both effectiveness and implementation outcomes could inform a future type-I stepped wedge cluster randomized control trial assessing the intervention's true potential.

Acknowledgements

The authors wish to acknowledge the patient and nurse participants who were essential to this study and the resulting improvements to patient care at the MUHC. These include nurse Emilie Bourbonnière, Melodie Brown, Mariko Duknic, Nancy Obas, and Luciana Ruppenthal. They also wish to acknowledge other clinicians and allied health professionals at the MUHC for their support of the project; Julie Bellingham and Chantal Burrell for their support in translating study documents; Costas Pexos and Jessica Lumia for supporting changes to the RISQ database; Gino Curadeau, Christian Machucha, Ellen Seguin, Vicky Lavazos, Jonathan Roger, Julia Slater, and Sharon Sandy for their ongoing support; and the I-Score patients committee for their assistance in developing study tools and procedures.

Funding: This study is funded by ViiV Healthcare.

Authors’ contributions: J.C. and N.K. are the co-principal investigators. J.C. is the study sponsor. B.Linthwaite, is the research coordinator and internal facilitator. L.R. is a nurse at the CVIS-MUHC. B.Linthwaite, J.C., N.K., and I.M. co-developed the analysis plan. B.Linthwaite, J.C., and N.K. were responsible for obtaining ethics approval. B.Linthwaite wrote the first draft of this article and the protocol for ethics review. B.Linthwaite, J.C., and N.K. collaborated to complete subsequent drafts. All authors commented and critically reviewed drafts of the article. All authors read and approved the final manuscript.

Conflicts of interest

J.C. has received consulting fees from ViiV Healthcare, Merck, and Gilead; research funding from ViiV Healthcare, Merck and Gilead; and payment for lectures from Gilead. N.K. has received consulting fees from ViiV Healthcare, Merck and Gilead; research funding from ViiV Healthcare and Gilead; and payment for lectures from Gilead. B. Lebouché has received consulting fees from ViiV Healthcare Merck and Gilead; research funding from Merck, Gilead, ViiV Healthcare; and payment for lectures from Merck, Gilead and ViiV Healthcare. All other authors declare that they have no competing interests.