Visual outcomes and treatment adherence of patients with macular pathology using a mobile hyperacuity home-monitoring app: a matched-pair analysis.

OBJECTIVE: We compared patients with neovascular age-related macular degeneration (nvAMD), diabetic macular oedema (DMO) and other macular pathologies testing their vision with the hyperacuity home-monitoring app Alleye to patients not performing home-monitoring regarding clinical outcomes and clinical management.
DESIGN: Matched-pair analysis.
SETTING: Retina Referral Centre, Switzerland. PARTICIPANTS: For each eye using Alleye, we matched 2-4 controls not using home-monitoring based on age, gender, number of previous intravitreal injections (IVI), best corrected visual acuity (BCVA) (Early Treatment Diabetic Retinopathy Study letters), central macular thickness (CRT) and time point of enrolment, using the Mahalanobis distance matching algorithm. We included 514 eyes (288 patients); 107 eyes with nvAMD using home monitoring and 218 controls not using home monitoring, 25 eyes with DMO (n=52 controls) and 40 eyes with miscellaneous conditions (n=72 controls). 173 eyes (33.7%) received no IVI during follow-up. MAIN OUTCOME MEASURES: Improvement of ≥5 letters, number of injection visits and treatment retention after correcting for differences in baseline characteristics with multivariate analyses.
RESULTS: The mean follow-up duration was 809 days (range 147-1353) and the mean number of IVI/year among treated eyes was 6.7 (SD 3.1). Mean age at baseline was 70.4 years (SD 10.9), BCVA was 77.6 letters (SD 11.6) and CRT was 263.6 µm (SD 86.7) and was similar between patients using and not using home monitoring. In multivariate analyses, patients using home monitoring had a higher chance to improve visual acuity by ≥5 letters (OR 1.67 (95% CI 1.01 to 2.76; p=0.044)) than controls. Treated eyes using home monitoring had less injection visits/year (-0.99 (95% CI -1.59 to -0.40; p=0.001)) and a longer treatment retention +69.2 days (95% CI 2.4 to 136.0; p=0.042). These effects were similar across retinal pathologies.
CONCLUSIONS: This data suggest that patients capable of performing mobile hyperacuity home monitoring benefit in terms of visual acuity and discontinue treatment less often than patients not using home monitoring. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first study assessing the impact of mobile hyperacuity monitoring on clinically relevant outcomes.

To conduct a randomised study comparing the management with home-monitoring versus usual care was not feasible and a matched-case analysis was used instead.

We attempted to mitigate the risk of bias by sampling several controls for each eye using the home monitoring.

We carefully matched controls on the basis of important clinical parameters that—if unbalanced between groups—could confound the analysis.

Introduction

Self-care increases personal health responsibility and promotes the empowerment of individuals’ involvement in management of their chronic diseases.1 The WHO has shortlisted self-care as health topic to maintain health, and cope with illness and disability.1 To date however, the management of many chronic eye diseases such as diabetic retinopathy, macular degeneration and glaucoma remains guided by data obtained during sporadic outpatient clinic visits.2 We often fail to capture the dynamic fluctuations in chronic eye disease that contain valuable data that could help to plan and individualise treatment.

Home monitoring via patient self-measurements provides a novel source of clinical data and not only reduces the need to attend the hospital but further allows the collection of high quality, structured data in an extramural setting for a personalised and targeted management. However, before implementing a self-monitoring programme, it is important to have evidence that it does no harm at the individual or population level.1 European and other national regulatory authorities such as the Food and Drug Administration in the United States of America (US FDA) have published medical device regulations to which self-measurement instruments need to comply.3 4

Two CE-marked and FDA cleared mobile apps for the home-monitoring of macular pathology are currently available; the mVT (Genentech USA) and the Alleye app (Oculocare medical Inc).5–7 The mVT implemented a shape discrimination task and tests 3 degrees, the Alleye—using a dot alignment task—tests 12 degrees of the central retina.8

In this paper, we compared patients with neovascular age-related macular degeneration (nvAMD), diabetic macular oedema (DMO) and other macular pathologies testing their vision with the Alleye app to patients on standard care, using a matched-case analysis.

Methods

Study design, setting and ethics

Within an ongoing prospective cohort study of patients starting home-monitoring with the mobile hyperacuity app Alleye, we enrolled all patients entering the study from September 2017 to June 2019 at the outpatient Medical Retina Service of the Eye Clinic of the Cantonal Hospital Lucerne. The control group was sampled retrospectively from patients screened during the same time period (September 2017 to June 2019) not willing to participate in the home monitoring but consenting to provide clinical data into this study.

Objectives

This study compared patients with nvAMD, DMO and other macular pathologies testing their vision with the hyperacuity home-monitoring app Alleye to patients not performing home monitoring in respect to differences in visual acuity, number of injections per year and follow-up duration. All patients were treated according to the clinics best practice guideline using a treat and extent treatment scheme. In exploratory analyses, we aimed to assess whether potential differences between these two groups overall could also be seen in three clinical subgroups: patients with nvAMD, DMO and patients with ≥60 Early Treatment Diabetic Retinopathy Study (ETDRS) letters at baseline.

Patient selection and matching

Patients with retinal pathology currently under treatment with anti-angiogenic therapy (ranibizumab (Lucentis) or aflibercept (Eylea)) of at least one eye, who had completed treatment induction qualified for inclusion if they presented with a best-corrected visual acuity (BCVA) of at least 35 ETDRS letters. Patients with a steroid intravitreal implant were not considered in this study. If both eyes were affected, both eyes were included in the study. Patients with a neurological or physical condition were excluded.

We performed individual matching9 using the Mahalanobis distance matching algorithm.10 For each eye using Alleye, we matched 2–4 control eyes on standard care based on age, gender, number of previous intravitreal injections (IVI), BCVA (ETDRS letters) and central retinal thickness (CRT). To avoid sampling bias in differences in follow-up duration, the eyes that were eligible as controls had to be included within 14 days of the enrolment date of the eye that performed the home monitoring. These parameters were considered the strongest possible confounders distorting the home-monitoring versus clinical outcome relationship. Matching was performed for three underlying retinal pathology groups separately: nvAMD, DMO and a third miscellaneous group. If an Alleye patient only had one eye with a retinal pathology, it received a healthy matched control eye.

Home monitoring setting

The details of home monitoring have been published elsewhere.11 In brief, after a training period involving repeated testing, patients were asked to perform a baseline assessment in the clinic and home monitoring. Patients performed the test while wearing their own glasses with each eye tested individually. Testing was either conducted on an iPod Touch (Apple) that was provided by the hospital or on the patients’ mobile device once the application was downloaded from Apple’s App or the Android Google Play store. In addition, all patients received an instruction manual for the application’s usage, and healthcare professionals provided technical assistance during clinical follow-up visits if needed. Patients were asked to conduct home monitoring twice weekly. The test was performed monocularly after having covered the non-tested eye.

The Alleye test is indicated for the detection and characterisation of metamorphopsia, a visual distortion in patients with nvAMD and in patients with DMO, as an aid in the monitoring of the progression of this condition in respect of metamorphopsia. It is intended to be used by patients who have the capability to regularly perform a simple self-test at home. The response to the Alleye test consists of two elements: a colour and a number. The colour indicates whether the patient was much worse (red); worse (yellow); equally good or better (green) compared with the last Alleye tests. The number indicates how many points the patient has set correctly. The maximum score is 100. If the test gives a red result three times in a row, the Alleye app will automatically inform the patient that occasional contact with the treating eye care professional is advisable. Technical specifications are available elsewhere (https://alleye.io/support/).

Data collection

In addition to the salient baseline characteristics, we recorded the number of IVI visits during the follow-up period and the follow-up duration for each participant; defined as the time interval between enrolment into the study and the last recorded clinical visit. The clinical characteristics at this event was also recorded. If a value of BCVA or CRT was missing at the last follow-up date, we recorded the values of the nearest previous visit. We additionally recorded reasons for discontinuation if they were stated in the electronic health record system. Follow-up was censored as of 16 November 2020.

Statistical analysis

All patients using Alleye and fulfilling the inclusion criteria were included. No formal sample size analysis was conducted. We calculated differences of BCVA and CRT from baseline to follow-up and calculated the proportion of participants gaining ≥5 letters from baseline to follow-up. For each eye and patient, we calculated the follow-up duration and the number of injections received during follow-up. We compared and tested differences in baseline and follow-up parameters between patients using hyperacuity home-monitoring versus usual care in univariate fashion, stratified for three clinical subgroups, nvAMD, DMO and a miscellaneous group.

For the dichotomous outcome ≥5 letters gain, we fitted a logistic regression model. For the two continuous outcomes number of injections/year and duration of retention (months of follow-up), we used two linear regression models. The statistical models accounted for the slight differences in baseline characteristics and were run in multivariate fashion correcting for participants’ age, female gender, BCVA at baseline, CRT at baseline and number of injections received previous to study entry. Because some participants provided both eyes to the analysis, we applied a clustered analysis, where each participant was considered as a cluster. This adjusted for the dependency of measurements in the fellow eye.

Besides an overall analysis, we conducted several stratified analyses. We assessed changes in visual acuity only within eyes receiving IVI at some point during follow-up and only within eyes≥60 letters at baseline, as this threshold was found to identify patients, where the Alleye test performed particularly well.7 Finally, we conducted stratified analyses within patients with nvAMD and DMO separately. Furthermore, in order to account for the fact that the potential length of follow-up period was different for each participant, we repeated the analysis in a subgroup of 195 participants providing a follow-up of 36 months (±2 months). Within that group, we assessed the likelihood of gaining ≥5 letters, the number of injection visits during the follow-up of 36 months and the changes of visual acuity and CRT from baseline.

Analyses were performed using the Stata V.16.1 statistics software package (StataCorp. 2019. Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC) and considered a p value of <5% as statistically significant.

Patient and public involvement

The development of the research question and outcome measures were developed on the basis of patients’ clinical priorities, but we did not involve patients in the design of this study. No patients were involved in the recruitment and the conduct of this study. Results from this study will be made available through the patient portal and information material provided by the Medical Retina Service of the Eye Clinic of the Cantonal Hospital Lucerne.

Results

Patients’ characteristics

This analysis included 514 eyes (172 eyes using home monitoring, 342 control eyes; 288 patients) comprising of 107 eyes with nvAMD using home monitoring and 218 controls not using home monitoring, 25 eyes with DMO (n=52 controls) and 40 eyes with miscellaneous conditions (n=72 controls). One hundred and seventy-three eyes (33.7%) received no IVI during follow-up (63 eyes of the home monitoring group and 110 from the control group). The mean follow-up duration was 809 days (range 147–1353). The mean age when entering the study was 70.4 years (SD 10.9), BCVA was 77.6 letters (SD 11.6) and CRT 263.6 µm (SD 86.7). The distribution of baseline characteristics was similar between those patients using home monitoring and those not and also similar within the three groups of macular pathologies (table 1).

Table 1

Comparing the clinical parameters between eyes using Alleye and matched controls, stratified for retinal pathology

Eyes assessed (n=514)	nvAMDAlleye (n=107)	Controls (n=218)		DMOAlleye (n=25)	Controls (n=52)		MiscellaneousAlleye (n=40)	Controls (n=72)		OverallAlleye (n=172)	Controls (n=342)
Parameters	Mean (SD)	Mean (SD)	P value	Mean (SD)	Mean (SD)	P value	Mean (SD)	Mean (SD)	P value	Mean (SD)	Mean (SD)	P value
Baseline
Age at baseline	74.9 (6.2)	75.4 (5.8)	0.476	63.5 (10.3)	63.8 (12.0)	0.915	59.5 (15.8)	61.7 (11.1)	0.391	69.7 (11.9)	70.8 (10.3)	0.282
Female gender	60.0%	60.0%	0.912	50.0%	50.0%	0.879	40.0%	40.0%	0.864	53.5%	53.5%	0.997
Previous IVI	9.8 (13.9)	10.3 (16.9)	0.791	11.8 (11.5)	5.1 (7.9)	0.004	6.9 (9.8)	4.5 (8.3)	0.172	9.4 (12.8)	8.3 (14.6)	0.383
Visual acuity baseline (letters)	77.8 (11.4)	76.5 (11.5)	0.338	79.2 (8.9)	78.8 (8.0)	0.844	78.3 (15.8)	78.8 (12.8)	0.856	78.1 (12.2)	77.3 (11.3)	0.470
Central retinal thickness baseline (µm)	248.3 (68.5)	256.9 (75.9)	0.323	258.5 (71.6)	298.4 (95.3)	0.068	306.4 (164.7)	259.1 (64.2)	0.033	263.3 (102.0)	263.7 (78.1)	0.962
Follow-up
Changes in visual acuity from baseline (letters)	−1.7 (9.5)	−3.2 (11.6)	0.247	2.4 (5.9)	−1.3 (7.7)	0.037	0.2 (9.6)	−0.4 (9.7)	0.754	−0.7 (9.2)	−2.3 (10.8)	0.087
Changes in central retinal thickness from baseline(µm)	−8.6 (54.8)	−5.6 (88.1)	0.747	18.6 (102.9)	−16.7 (81.9)	0.108	−38.7 (148.5)	8.8 (82.2)	0.031	−11.6 (93.2)	−4.3 (86.1)	0.373
Follow-up duration (months)	28.4 (8.4)	27.4 (9.3)	0.348	30.4 (7.2)	24.2 (11.4)	0.015	25.4 (11.0)	22.4 (9.5)	0.133	28.0 (9.0)	25.9 (9.9)	0.017
Mean IVI/month	0.4 (0.3)	0.4 (0.3)	1.000	0.3 (0.2)	0.3 (0.3)	1.000	0.2 (0.3)	0.4 (0.4)	0.007	9.6 (10.3)	10.4 (9.8)	0.407

Continues variables were tested using independent t-tests, dichotomous variables were tested using the χ2 or Fisher’s exact test, were appropriate.

Statistically significant differences at baseline are highlighted (bold italic). Differences in outcomes (lower part of the table) are highlighted (bold)

IVI, intravitreal injection.

For the three outcomes assessed, the overall analyses showed positive effects for home monitoring. The summary of results is shown in table 2.

Table 2

Summary of results from multivariate analyses, overall and stratified for diagnostic subgroups (neovascular age-related macular degeneration (nvAMD), diabetic macular oedema (DMO)) and for eyes with ≥60 Early Treatment Diabetic Retinopathy Study letters visual acuity (including patients with nvAMD and DMO) at the baseline examination

Outcomes	OR (95% CI)	P value
Gaining≥5 letters at the follow-up
Overall	1.67 (1.01 to 2.76)	0.044
nvAMD	1.60 (0.84 to 3.07)	0.155
DMO	2.62 (0.69 to 10.03)	0.159
≥60 letters at baseline	1.72 (1.04 to 2.83)	0.035
	Mean difference (95% CI)
Number of injection visits per year
Overall	−0.99 (−1.59 to −0.40)	0.001
nvAMD	−0.61 (−1.39 to 0.17)	0.124
DMO	−1.14 (−2.27 to −0.01)	0.048
≥60 letters at baseline	−0.74 (−1.34 to −0.13)	0.018
Follow-up duration (days)
Overall	69.2 (2.4 to 136.0)	0.042
nvAMD	33.0 (−48.6 to 114.7)	0.426
DMO	138.2 (−32.0 to 308.3)	0.108
≥60 letters at baseline	67.9 (−0.8 to 136.7)	0.053

Changes in visual acuity

Among patients using home monitoring the number of eyes gaining ≥5 letters at the follow-up was 43/172 (25.0%) and 65/342 (19.0%) among eyes on standard care. Patients using home monitoring had a higher chance of an improvement of ≥5 letters (OR 1.67 (95% CI 1.01 to 2.76; p=0.044)) than controls when correcting for confounding due to baseline differences in age, female gender, BCVA, CRT and number of previous injections.

Within the subgroup of eyes with nvAMD, 66/325 eyes (20.3%) improved ≥5 letters and within the group of DMO, 17/77 eyes (22.1%) reached this outcome. In both, the nvAMD (OR 1.60 (95% CI 0.84 to 3.07; p=0.155)) and the DMO groups (OR 2.62 (95% CI 0.69 to 10.03; p=0.159)), the use of home monitoring was associated with a higher likelihood of improving by ≥5 letters; however, these associations did not reach statistical significance. Within the subgroup of participants with BCVA at baseline ≥60 ETDRS letters (n=467 eyes; 90.9%), the likelihood of an improvement of ≥5 letters when using home monitoring was slightly higher (OR 1.72 (95% CI 1.04 to 2.83; p=0.035)).

Number of visits

The mean number of IVI/year among treated eyes was 6.7 IVI/year (SD 3.1). Among those eyes using home monitoring (n=109), the mean number of injections was 6.0 (SD 3.2) and 7.0 (SD 3.0) among eyes not using home monitoring (n=232). Treated eyes using home monitoring had less injection visits per year (−0.99 injection visits (95% CI −1.59 to −0.40; p=0.001)) when correcting for baseline differences in age, female gender, BCVA, CRT and number of previous injections. Within the subgroup of eyes with nvAMD receiving IVI (n=225) and within the group of eyes with DMO (n=55), these associations were similar (nvAMD: −0.61 (95% CI −1.39 to 0.17; p=0.124); DMO: −1.14 (95% CI −2.27 to −0.01; p=0.048)). Within the subgroup of eyes with BCVA ≥60 ETDRS letters at baseline, the corresponding values were −0.74 (95% CI −1.34 to −0.13; p=0.018). When assessing interaction in this group, eyes of patients using home monitoring received statistically more injections (+2.79 injections/year (95% CI 0.65 to 4.93; p=0.011)) than controls.

Follow-up durations

In the eyes of patients performing home monitoring, we found a longer duration of follow-up of +69.2 days (95% CI 2.4 to 136.0; p=0.042) compared with eyes not using home monitoring, when correcting for baseline differences in age, female gender, BCVA, CRT and number of previous injections. Within the nvAMD group, the difference was smaller (+33.0 days (95% CI −48.6 to 114.7; p=0.426)) and notably higher within the DMO group (+138.2 days (95% CI −32.0 to 308.3; p=0.108)) without reaching statistical significance. Within the subgroup of eyes with BCVA ≥60 ETDRS letters at baseline, the corresponding values were +68.4 days (−0.9 to 137.7; p=0.053).

Subgroup analyses in patients providing 36 months of follow-up

Among patients using home monitoring, the number of eyes gaining ≥5 letters was higher (OR 2.92 (95% CI 1.21 to 7.04); p=0.017). Patients performing home monitoring had less injection visits during the 36 months of follow-up than controls. This difference did not reach statistical significance (−1.59 (95% CI −4.18 to 0.99); p=0.224). Also, CRT decreases were more pronounced in the home monitoring group than in controls, without reaching statistical significance (−18.9 µm (95% CI −52.4 to 14.7); p=0.268). However, patients using home monitoring had a significantly higher BCVA after 36 months of follow-up than patients in the control group (4.80 ETDRS letters (95% CI 1.04 to 8.57); p=0.013).

Discussion

Main findings

In this matched cases study, analysing clinical outcomes during a mean follow-up period of more than 2 years, we found a consistent pattern of positive effects of hyperacuity home monitoring including an increased likelihood of clinically relevant vision gain, lower number of IVIs and extended treatment retention. The willingness to participate in a home monitoring programme was associated with beneficial clinical outcomes through mechanisms that are not completely clear yet.

Results in context of the existing literature

The magnitude of effects of home monitoring in this study—that predominantly enrolled patients with a macular pathology who received IVI treatment—were similar to those of a large randomised study assessing whether hyperacuity home monitoring in combination with tele-monitoring, resulted in higher BCVA when choroidal neovascularisation in nvAMD was detected.12 13 The logic behind the HOme Monitoring of the Eye (HOME) study was that patients would benefit from early detection, since timely intervention—when deterioration of visual acuity is less pronounced—would lead to better long term outcomes. Indeed, the median difference in the decline of visual acuity at onset of detected choroidal neovascularization between randomised groups in the HOME study was five ETDRS letters. Our paper demonstrates that patients, who, even once a macular pathology was diagnosed and treated, benefited from home monitoring. The home monitoring group had about a twofold higher likelihood of gaining ≥5 letters than the group not using home monitoring. The patients included in our study followed a treat and extend (TAE) treatment scheme that was reassured with the Alleye home monitoring (TAE reassured by Alleye: TAERA). While extensions were decided based on the clinical course and optical coherence tomography assessments, patients performing home monitoring were encouraged to report to the clinic when the Alleye test indicated an increase of metamorphopsia. While overall patients in the home monitoring group needed less IVI, we observed a trend towards more IVI in eyes with a baseline BCVA of 60 ETDRS letters or more using the home monitoring. While the mechanisms leading to a higher treatment intensity in this particular group of patients remain ill understood, we speculate that home monitoring increased disease awareness in our patient cohort. Notably, patients using home monitoring remained longer in treatment than patients who did not.

Strength and limitations

We employed a pragmatic matched-case analysis on the basis of a previously initiated cohort study of patients using Alleye hyperacuity monitoring at home.7 In the absence of the possibility to conduct a randomised study comparing the management with Alleye versus usual care, this design allows the exploration of this association without exposing patients to any increased risks. However, a matched-case analysis is prone to several biases. First and foremost, selection of control subjects can be problematic. We attempted to mitigate this risk by sampling several controls for each eye using the home monitoring. Moreover, we carefully matched controls on the basis of important clinical parameters that—if unbalanced between groups—could confound the analysis. Indeed, the groups were fairly similar as shown in table 1. Moreover, we performed multivariate analyses including the matching parameters to further counteract confounding. However, even when employing all the recommended measures for a valid analysis, we cannot rule-out that residual confounding occurred. For example, patients agreeing to perform home monitoring might be generally more motivated to adhere to therapy. Despite this, and also considering that our results were consistent within various subgroups we believe that confounding was not a significant issue.

Implications for research

After assessing validity, reliability and the diagnostic properties of the mobile hyperacuity app Alleye,6 7 14 this is the first study investigating the relationship between home monitoring use and clinical outcomes.15 From a methodological viewpoint, confirmation from independent research is certainly warranted. On the other hand, we propose to envision a series of studies assessing the effects and heterogeneity of mobile hyperacuity monitoring within difference clinical settings.16 Real-world evidence on the one hand, assessing the local effects within a specific care setting, with cost-effectiveness analyses on the other hand together may contribute to a better understanding as to where and how home monitoring works best and why.

Implications for practice

In many fields of ophthalmology we see a benefit from telemedicine and home monitoring.2 17 A recent expert survey assessing the barriers of the implementation of digital health into clinical practice identified reimbursement of modern remote services to be an important facilitator.18 Quite apart from the home monitoring of macular pathology, there are several other exciting projects involving the remote management of ocular conditions such as glaucoma that are under way.8 19 20 Implementing change to well-cemented clinical regimes is notoriously difficult. Through tackling the challenges of chronic eye management during the SARS-CoV-2 pandemic, we furthered our understanding as to how digital solutions can promote the safe and effective provision of healthcare remotely.17 Recently, a group of researchers based at Moorfields Eye Hospital in London using the usefulness of Alleye home monitoring during the SARS-CoV-2 pandemic showed that four out of five patients presenting with alarms had clinical progression and two out of three patients required immediate intravitreal therapy.21

Conclusions

Our data suggest that patients capable of performing mobile hyperacuity home monitoring benefit in terms of visual acuity gains and discontinue treatment less often than patients who are not using home monitoring. Considering that the mobile hyperacuity home monitoring is available and ready to use, adoption of such technology should be encouraged in clinical environments. Ideally, such implementation is accompanied by research to generate the necessary real-world evidence so potential reservations regarding the use of such digital solutions can be addressed and alleviated.