The State of Patient-Reported Outcome Measures for Pediatric Patients with Inherited Retinal Disease.

Patient-reported outcome measures (PROMs) are questionnaires that assess health outcomes meaningful to the patient. PROMs have multiple applications, such as supporting clinicians' decision-making for patient care, understanding the impact of disease on patient functioning, and evaluating the efficacy of therapeutics. Though PROMs were developed for various eye conditions, no PROM was tailored to pediatric patients with inherited retinal disease (IRD). Hence, a literature search was conducted using MEDLINE and Embase to identify PROMs potentially relevant to this patient population. This review evaluated selected pediatric PROMs against the US Food and Drug Administration (FDA) guidelines and found restricted use in the context of IRD. As there is a need for PROMs tailored to pediatric patients with IRD, we provide a perspective on applying the International Society for Pharmacoeconomics and Outcomes Research and FDA standards on the development of PROMs specific to IRD.

Key Summary Points

Introduction

Inherited retinal diseases (IRDs) are a group of hereditary diseases of the retina that can result in progressive photoreceptor degeneration, subsequent severe vision loss, and blindness in children and adults. IRDs underlie an expressive economic and psychosocial burden. In 2019 alone, IRDs contributed approximately US $14,685.1–37,003.8 million in economic and well-being costs in the USA and Canada [1]. After deciphering the genetic basis of many IRDs, gene-specific therapies are being developed to restore some sight and slow or halt disease progression. A recent ocular gene replacement therapy, voretigene neparvovec-rzyl, was the first to be approved by the US Food and Drug Administration (FDA), European Medicines Agency, and Health Canada for patients with RPE65-Leber congenital amaurosis (LCA), an early and severe form of IRD. Since then, numerous clinical trials for other IRDs therapies have been underway [2]. In principle, novel interventions should aim to minimize the impact of disability on daily living, which can only be measured by patient-reported outcome measures (PROMs). This could not be thoroughly assessed in patients undergoing these trials as IRD-specific tools were not available.

Patient-reported outcome measures are self-reported questionnaires measuring health outcomes from the patient perspective. The FDA has defined specific features of PROM development [3, 4]. A validated PROM should be created by incorporating an in-depth qualitative analysis of patient input (i.e., interviews or focus groups) and thorough quantitative psychometric analysis to establish the measure’s validity, interpretability, reliability, and ability to detect a change in the target population. When coupled with other clinical tests, well-designed PROMs can holistically measure a therapeutic impact on a patient’s daily life [4-6].

To date, many clinical trials for voretigene neparvovec-rzyl and other ophthalmic therapies have used the National Eye Institute-Visual Function Questionnaire-25 (VFQ-25) as a secondary outcome measure, among others. The VFQ-25 is a widely used PROM, stated to assess vision function [7]; however, it is not IRD-specific. That said, until recently IRD-specific PROMs did not exist. The Michigan Retinal Degeneration Questionnaire (MRDQ) and Michigan Vision-related Anxiety Questionnaire (MVAQ) were recently developed adult IRD-tailored PROMs that assess vision ability and associated psychosocial health, respectively [8, 9]. The MRDQ generates an ability score from adult patient-reported visual function in domains representative of physiological visual function pathways and thereby can record a patient’s perspective on emerging therapeutics for these domains by using theta scores (item response theory) derived using a graded response model. The MRDQ was originally validated in adults, limiting its current use in the pediatric population as infants, children, and adolescents are at vastly different developmental stages than adults [10]. A patient-centric PROM requires consideration of the characteristics and perspectives of the age group. PROMs for pediatric patients should reflect their language level and the relevancy of experiences encapsulated by the questions to their age group as well as their condition.

There is an increasing need to assess the effect of therapies for pediatric patients with IRDs using PROMs to better understand changes observed from the patient perspective. This present review aims to evaluate if existing PROMs for pediatric patients with vision loss can be used for the IRD population and provide a perspective for developing PROMs for pediatric patients.

Methods

Search Methods for Identifying Studies

Patient-reported outcome measures (PROMs) tailored to pediatric ophthalmic conditions were identified in two ways. First, this review included the ophthalmic PROMs described in Tadic et al.’s [11] systematic review. Second, additional PROMs were identified from MEDLINE and Embase databases by adapting Tadic et al.’s [11] search strategy to cover PROMs developed after 2013 that could be potentially used for inherited retinal disease (IRD) and its substituent conditions. The strategy was verified by consultation with an information scientist (Table S1, Supplementary Material). The search was restricted till 2013 to avoid overlap with Tadic and colleagues’ search [11]. The citations were exported to the Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia; http://www.covidence.org) to de-duplicate and complete multilevel screening [12].

Study Selection

Studies were selected after two levels of screening (Fig. 1). First, title/abstract screening was conducted to remove papers if they were (a) focused on non-ophthalmic diseases, on a non-IRD ophthalmic disease, or not focused on visual impairment as a whole; (b) were gray literature, including conference abstracts, non-peer-reviewed text, or records only on the preliminary phases of PROM development; (c) PROM development or validation was not the primary objective or non-primary research, including reviews; (d) PROMs were not validated or made in English (transcultural adaptions into English were kept); and (e) participants were over 18 years of age. Then full-text screening was done with the title/abstract eligibility criteria, additionally filtering out studies included in Tadic et al.’s [11] systematic review.

Fig. 1

Flow diagram of literature screening

Data Extraction and Descriptive Analysis

Data extracted from each study included study description (i.e., age of participants, country, year, eligibility criteria, number of patients with IRD for each phase of PROM development), version of PROM (i.e., original or revised), and PROM description (i.e., number of items, questionnaire respondent, purpose, and subscale(s)/domains). The study characteristics are summarized in Table 1. Criteria set by the US FDA were used to evaluate each PROM’s applicability to pediatric patients with IRD (summarized in Table 2). A validated PROM has undergone an in-depth qualitative analysis of patient input (i.e., interviews or focus groups) and thorough quantitative psychometric analysis to establish the measure’s validity, interpretability, reliability, and ability to detect a change in the target population [4, 13]. As such, a representative sample is required for PROM development [3]. The study characteristics were qualitatively explored as part of the synopsis of each paper to identify the merits and limitations of the PROM’s applicability in pediatric patients with IRD.

Table 1

Characteristics of included ophthalmic patient-reported outcome measures for pediatric patients with vision impairment

PROMs	Country and year of publication(s)	Original or reviseda	Number of items (age range, respondent)	Number (%) of patients with IRD in sample	Purpose/construct	Subscale(s)/domain(s)
Children’s Visual Function Questionnaire (CVFQ) [14, 15]	USA, 2004–2007	Original	35 (0–3 years, self-report); 40 (3–7 years, self-report)	29*/403 (7%) for pilot testing [14]; 58*/397 (15%) in < 3 years and 80*/376 (21%) in ≥ 3 years for validation [14]; 0*/194 were IRD for further validation [15]	To measure vision-specific QoL in children with VI	General health; general vision; competence; personality; family impact; treatment
Cardiff Visual Ability Questionnaire for Children (CVAQC) [16]	UK, 2010	Original	25 (5–18 years, self-report)	Not given	To assess VA in children and young people VI	Education; near vision; distance vision; getting around; social interaction; entertainment; sports
Impact of Visual Impairment for Children (IVI_C) [17, 18]	Australia, 2008–2011	Original	23 (8–18 years, self-report)	Not given	To measure the effect of impaired vision on QoL in children with VI	No subscale(s) reported
Pediatric Eye Questionnaires (PedEyeQ) [20–24]	USA, 2017–2021	Original	40 (5–11 years, self-report); 39 (12–17 years, self-report); 29 (0–4 years, parent-proxy); 39 (5–11 years, parent-proxy); 42 (12–17 years, parent-proxy); 35 (parent self-report)	12**/124 (9.6%) in 0–4 years (parent-proxy), 12**/117 (10.3%) in 5–11 years (self-report and parent-proxy), 13**/87 (14.9%) in 12–17 years (self-report and parent-proxy) for content development [20]; 55*/444 (12.4%) in 0–17 years (self-report and parent-proxy) for pilot testing [21]; 5/45 (11.1%) in 0–4 years (parent-proxy), 5/40 (12.5%) in 5–11 years (self-report and parent-proxy), 3/22 (13.4%) in 12–17 years (self-report and parent-proxy) for validation [22]	To assess eye-related quality of life in VI children	(5–11 years and 12–17 years, self-report): functional vision; bothered by eyes/vision; social; frustration/worry (0–4 years, parent-proxy): functional vision; bothered by eyes/vision; social (5–11 years and 12–17 years, parent-proxy): functional vision; bothered by eyes/vision; social; frustration/worry; eye care (parent self-report): impact on parent/family; worry regarding child’s eye condition; worry regarding child’s self-perception and interactions; worry regarding child’s visual function
Vision-related Quality of Life of Children (VQoL_C) and Vision-related Quality of Life of Children (VQoL_YP) [29]	UK, 2011–2021	Revised	20 (8–12 years, self-report); 22 (13–17 years, self-report)	7*/12 (75%) in 7–9 years, 12*/17 (70.6%) in 16–19 years for content development [29]; 8*/12 (66.7%) in 7–10 years, 9*/16 (56.3%) in 13–18 years for pretesting [29]; 15*/26 (57.7%) in 8–12 years, 18*/23 (78.3%) in 13–17 years for pilot testing [29]; 56*/87 (64.4%) in 7–13 years, 50*/73 (68.5%) in 13–18 years for validation [29]	To measure vision-specific QoL in children with VI	No subscale(s) reported
Functional Vision Questionnaire for Children (FVQ_C) and Functional Vision Questionnaire for Young People (FYQ_YP) [32]	UK, 2011–2021	Revised	28 (8–12 years, self-report); 38 (13–18 years, self-report)	9*/12 (75%) in 6–9 years, 12*/17 (70.6%) in 16–19 years for content development [32]; 8*/12 (66.7%) in 7–10 years, 9*/16 (56.3%) in 13–18 years for pretesting [32]; 71*/113 (62.8%) in 7–13 years, 68*/96 (70.8%) in 13–18 years for pilot testing and validation [32]	To assess the functional impact of VI on activities of daily living in children and young people	No subscale(s) reported

PROM patient-reported outcome measure, QoL quality of life, VI vision impairment

* = IRD + retinal disease; ** = IRD + retinal disease + optic neuropathy

aRevised = New edition of PROM released since initial validation

Table 2

Summary of criteria to evaluate applicability of vision impairment PROMs based on FDA guidelines

Criteria	Definition
Validity	The capacity of the PROM to describe the trait it intends to measure  e.g., content-related, criterion-related, predictive
Reliability	The consistency of the measurements taken by the PROM  “How the person functions” in regards to their vision e.g., test–retest, internal consistency, inter-interviewer reproducibility
Sensitivity	The instruments’ ability to detect a change in the population  e.g., effect size, standard error
Representative populationa	A PROM should be developed with the target population for the target population  In the case of IRD, there are many subtypes and so a representative sample for PROM development consisting of IRD’s three electroretinogram subtypes (rod-cone dystrophy, cone/cone-rod dystrophy, and macular dystrophy)

PROM patient-reported outcome measure

aAdditional criteria set out by the US Food and Drug Administration (FDA)

This article is based on previously published studies and does not contain any new studies with human participants or animals performed by any of the authors.

Results

Study Selection

Many PROMs are available for pediatric patients with different ophthalmic conditions [11]. There were no IRD-specific PROMs identified for pediatric patients; however, the PROMs described is this section have been developed for general visual impairment and may have included some patients with IRD in the making. That said, in the absence of IRD-specific PROMs, we focused on available tested and validated PROMs tailored to visual impairment (VI) in pediatric patients (Table 1).

Children’s Visual Function Questionnaire (CVFQ) [14, 15]

First developed by Felius, Birch, and colleagues in 2004, the Children’s Visual Function Questionnaire (CVFQ) covers all concerns young children (at most 7 years of age) have related to their vision impairment [14, 15]. The initial PROM combined several behavioral and visual function questionnaires alongside clinician input [14]. After pilot testing in patients, it was divided into two measures for two age groups (less than 3 years and 3–7 years) based on response frequency; these drafts were tested in a second set of patients. Factor analysis showed the presence of multiple subscales, which were defined as Competence, Personality, Family Impact, and Treatment Difficulty. The CVFQ was further validated in pediatric patients by assessing its sensitivity to different patient situations: unilateral vs. bilateral disease (i.e., cataracts), different severity of visual impairment (i.e., retinopathy of prematurity patients grouped by acuity), the difficulty of treatment regimen (i.e., optical, surgery, surgery + optical, and surgery + optical + occlusion), and alternative treatments for the same condition (i.e., intraocular lens, and aphakic contact lenses) [15]. Meaningful differences were found between the predefined groups, solidifying CVFQ’s importance in these types of clinical trials. Both studies were informative, but the measure’s development lacked visual impairment patient input when developing items. Additionally, 7–21% of the samples in the item development phases had IRD or other unspecified forms of retinal disease, while no patients had IRD in the validation phase—a critical yet missing piece to conclude on the sensibility to a PROM’s target group. Outside of patients with IRD, this tool still has significant validity for use in patients with a cataract.

Cardiff Visual Ability Questionnaire for Children (CVAQC) [16]

There was a need to assess pediatric patients’ (aged 5–18 years) difficulty in performing activities of daily living and track changes in difficulty performing these activities after rehabilitation. Khadka and colleagues [16] developed the Cardiff Visual Ability Questionnaire for Children (CVAQC) to meet this need and found no differential item functioning by age. Several subscales are defined in the measure: Education, Near vision, Distance Vision, Getting Around, Social Interaction, Entertainment, and Sports. The measure showed high measurement precision (person separation 2.28 and reliability 0.84; item separation 6.20 and reliability 0.97), and test–retest reliability (interclass correlation of 0.89 for persons (95% CI 0.80–0.94) and 0.94 for six items (95% CI 0.87–0.97)). However, the study is limited as it does not go into much depth on how the items were derived, severely VI patients were excluded from the psychometric validation phase, and patient breakdown by VI conditions is not given, so the IRD sample is undeterminable. Many PROMs following identified CVAQC as a precedent for measuring a change in patient experience following an ophthalmic intervention and is informative for PROM development.

Impact of Vision Impairment on Children (IVI_C) [17, 18]

Cochrane and colleagues (2008) aimed to assess the impact of VI on visually impaired patients (aged 8–18 years) and their caregivers, especially given that most measures developed then were solely based on caregiver or clinician-researcher input and did not include the patient perspective [17, 18]. An adult version of the Impact of Vision Impairment on Children (IVI_C [19]) PROM existed; however, it was not adapted to this age group; instead, the IVI_C was developed de novo. Focus groups were conducted to gain input from patients and other relevant stakeholders (i.e., teachers, caregivers, or specialists) [17]. Items developed from the qualitative portion were validated by Rasch analysis, yet no subscales were defined [18]. However, the inclusion criteria excluded patients with low VI (0.3 logMAR or less), and patient breakdown by VI condition is not given, so the IRD sample is undeterminable. Outside of patients with IRD, the measure showed particular use in other patients with low vision and near blindness.

Pediatric Eye Questionnaires (PedEyeQ) [20–24]

Holmes, Hatt, and colleagues (2021) [23] wanted to assess pediatric patients’ health and well-being across the entire spectrum of eye disease. They conducted concept elicitation interviews based on methods from their previous interview-based studies with different VI patient groups [25-28]. Through a thematic analysis, they identified common elements between the participants, which they used to create the first draft of the Pediatric Eye Questionnaires (PedEyeQ) PROM [20, 21]. There were four PROMs made for the VI patients, two sets for the groups of children aged 5–11 years and 12–17 years (each group having a patient self-report and a caregiver-report measure). The PROMs were then validated where the following domains were identified: child PedEyeQ domains are functional vision, bothered by eyes/vision, social, frustration/worry; and proxy PedEyeQ domains are functional vision, bothered by eyes/vision, social, frustration/worry, eye-care [22]. While there was a breakdown of IRD conditions given for different PROM development phases [20-22], only 9.6–14.9% of participants from the samples had IRD.

Vision-Related Quality of Life of Children (VQoL_C) and Vision-Related Quality of Life of Children (VQoL_YP) [29]

The Vision-related Quality of Life of Children (VQoL_C) and Vision-related Quality of Life of Children (VQoL_YP) recently developed by Rahi, Tadić, and colleagues (2021) are age-specific extensions of their 2011 VQoL_CYP [30, 31]. These tools were developed to assess all impacts of VI on visually impaired pediatric patients (aged 8–12 years, 13–17 years) on the basis of their perspective, otherwise known as vision-related quality of life (VRQoL). The strength of this PROM is its age-specific extensions. All editions were validated in the pediatric VI group. The studies had a large proportion of retinal patients (56.3–78.26% depending on the study phase) [32]; the number of patients with the inherited type was not specified. Additionally, they excluded participants with visual acuity in the better eye of logMAR ≤ 0.48 [31].

Functional Vision Questionnaire for Children (FVQ_C) and Functional Vision Questionnaire for Young People (FYQ_YP) [32]

Again by Rahi, Tadić, and colleagues (2021), the Functional Vision Questionnaire for Children (FVQ_C) and Functional Vision Questionnaire for Young People (FVQ_YP) are age-specific extensions of the FVQ_CYP from 2011 [30, 33]. The original and its extensions were developed to assess a visually impaired child’s perspective of their functional vision (aged 8–12 years, 13–18 years). All editions for this have been validated by Rasch analysis. The study had many patients with any retinal disease (56.3–75% depending on the study phase) [32]; the number of patients with the inherited type was not specified. As well, participants were excluded if the visual acuity in their better eye was logMAR ≤ 0.48 [33].

Discussion

Why Use PROMs Tailored to the Condition?

Patient-reported outcome measures can assess the impact of a condition on activities of daily living. The FDA [34] designates value in using an appropriate PROM in a therapeutic trial to highlight clinically meaningful differences in patients’ health and visual outcomes, where a general PROM may not have that capacity [35, 36]. General PROMs are multistate, grouping several conditions to assess health outcomes in general (e.g., general vision). This can result in floor or ceiling effects, where responses are pooled to extremes of a score distribution [37].

Current Status of PROMs in Inherited Retinal Disease Clinical Trials

Many past or ongoing gene therapeutic clinical trials included patients with IRD, specifically for retinitis pigmentosa, choroideremia, achromatopsia, LCA, Usher syndrome, X-linked retinoschisis, and Stargardt disease [38]. However, most of the peer-reviewed studies published did not include PROM results or any evaluation of the patient experience [5]. One study included the VFQ-25 [39], but as discussed earlier, the FDA states a validated PROM should be developed in the target population [4, 13], and so while VFQ-25 does have its strengths, it was not tailored to the pediatric IRD population.

New PROMs for Adult Patients with IRD

The Michigan Retinal Degeneration Questionnaire and Michigan Vision-related Anxiety Questionnaire are patient-reported outcome measures for adult patients with IRD to assess their vision ability and associated psychosocial health, respectively [8, 9]. These IRD-specific measures will begin to be incorporated into clinical trials studying voretigene neparvovec-rzyl as part of their outcomes and other observational studies. The use of MRDQ and MVAQ in a pediatric sample lacks validation. Studies are in progress to fulfill this need to holistically evaluate the efficacy of an intervention in children with IRD.

Perspectives on Pediatric PROM Development

Creating PROMs is iterative [3], and the product is held to the same rigorous standards as other outcome measures in clinical trials [13, 40]. The FDA and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) have created their own methods for designing a PROM [3, 10, 13, 40]. Both groups emphasize the importance of the PROM showing adequate validity and reliability. Thus far, no PROMs have been made for pediatric patients with IRDs, so the ISPOR guidelines are summarized here in the context of pediatric patients.

Determine the Construct

The construct or patient-reported outcome (PRO) is the concept or outcome of interest as defined by the patient [13]. This can be related to the disease, such as assessing condition-specific quality of life, functional ability, or an intervention for the patient group. Unlike adults, the pediatric group carries vast developmental differences [10]. The selection of age groups is relatively arbitrary, which is why this varies between studies. One solution toward more uniformity could be to refer to ISPOR’s guidelines (less than 5 years, 5–7 years, 8–11 years, and 12–18 years) [10].

Concept Elicitation and Analysis

Once a construct is defined, the researchers need to create items for the first draft of the PROM. First, an interview or focus-group guide is developed to lead interviews [13]. The guide itself will help elicit ideas, experiences, and concerns the patients have about their condition, which will be used to create the PROM. The design of the guide can be based on the literature or focus groups/interviews with stakeholders for the PROM (such as patients and relevant clinicians) [3, 13]. However, it is highly recommended to include the patient input into the guides as they are the intended group for the measure in development and to ensure construct validity [41]. In the case of infants and young children (less than 6 years), the majority of the concepts will come from the caregiver [10]. Between 6 and 12 years, the reliability and validity of the child’s responses can be variable, a problem not exclusive to PROMs, so incorporating both the child’s and caregiver’s responses is recommended. The focus group and interview transcripts should be thematically analyzed (i.e., looking for commonalities that can be used to create the questions in the interview/focus group guides) [13]. Once these guides are created, they can be administered to larger groups of patients to develop that repertoire of experiences needed to make the first draft of the PROM. Again, these experiences need to be thematically analyzed.

Pilot the Draft Instrument

A draft instrument should be ready by this stage. As with the previous stages, depending on the age group, the draft should be designed with the respondent in mind [42]. For caregiver-proxy PROMs, where the caregiver responds on behalf of the pediatric patient, the vocabulary can be more complex than patient self-report PROMs, but children’s PROMs should always have age-appropriate language. Following is one last stage of qualitative improvement before psychometric validation. The instrument can be administered in a small group of patients, where each participant should consider each question for relevance and comprehensibility [13]. This can be done through cognitive interviews, an evidence-based, qualitative method of improving and validating questionnaires, which uses the respondent’s cognition to identify incorrect or difficult items [42]. Once pilot-tested, the final draft of the PROM is ready for quantitative validation.

Psychometric Validation

The final draft of the PROM should be tested in a large sample of patients dependent on the appropriate sample size calculations and characteristics of the population. As such, in the case of orphan diseases like IRDs, the sample size calculations may be forgone, given the uncommonness of the condition in the general population [37]. Many different techniques exist to quantitatively assess the validity and reliability of the PROM [40]. This depends on the complexity of questions and so is at the researcher’s discretion. An important marker to identify in patient populations such as IRDs is the presence of floor or ceiling effects. The distribution of responses should have some level of correlation to the physiological impairment the patient faces [5]. Additionally, test–retest reliability, defined as the stability of scores over time when no change is expected in the concept of interest test–retest reliability, should be established for the questionnaire [43].

Conclusion

At the time of writing this review, there are no tailored patient-reported outcome measures available to assess the impact of IRD on pediatric patients; most pediatric tools are designed to assess the impact of general visual impairment. Outcome measures used for a specific condition should be rigorously developed with mixed methods input from its target condition and age cohort as advised by FDA and ISPOR. As such, the use of general visual impairment PROMs is not recommended for garnering significant insight for the purposes of clinical decision-making and evaluating the efficacy of therapies. This review establishes the need to develop PROMs for pediatric patients with IRDs.

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 134 kb)

There is an increasing need to assess meaningful health outcomes in pediatric patients with inherited retinal disease (IRD), especially with the development of gene therapies for these conditions.

There are no published patient-reported outcome measures (PROMs) that have been tailored to patients with IRD.

Evaluation of “general pediatric vision” PROMs against the US Food and Drug Administration (FDA) guidelines has shown low applicability for pediatric patients with IRD.

A need to develop PROMs tailored to pediatric patients with IRD has been identified.