Positive Outcomes: Validity, reliability and responsiveness of a novel person-centred outcome measure for people with HIV.

OBJECTIVES: Despite successful treatment, people living with HIV experience persisting and burdensome multidimensional problems. We aimed to assess the validity, reliability and responsiveness of Positive Outcomes, a patient-reported outcome measure for use in clinical practice.
METHODS: In all, 1392 outpatients in five European countries self-completed Positive Outcomes, PAM-13 (patient empowerment), PROQOL-HIV (quality of life) and FRAIL (frailty) at baseline and 12 months. Analysis assessed: (a) validity (structural, convergent and divergent, discriminant); (b) reliability (internal consistency, test-retest); and (c) responsiveness.
RESULTS: An interpretable four-factor structure was identified: 'emotional wellbeing', 'interpersonal and sexual wellbeing', 'socioeconomic wellbeing' and 'physical wellbeing'. Moderate to strong convergent validity was found for three subscales of Positive Outcomes and PROQOL (ρ = -0.481 to -0.618, all p < 0.001). Divergent validity was found for total scores with weak ρ (-0.295, p < 0.001). Discriminant validity was confirmed with worse Positive Outcomes score associated with increasing odds of worse FRAIL group (4.81-fold, p < 0.001) and PAM-13 level (2.28-fold, p < 0.001). Internal consistency for total Positive Outcomes and its factors exceeded the conservative α threshold of 0.6. Test-retest reliability was established: those with stable PAM-13 and FRAIL scores also reported median Positive Outcomes change of 0. Improved PROQOL-HIV score baseline to 12 months was associated with improved Positive Outcomes score (r = -0.44, p < 0.001).
CONCLUSIONS: Positive Outcomes face and content validity was previously established, and the remaining validity, reliability and responsiveness properties are now demonstrated. The items within the brief 22-item tool are designed to be actionable by health and social care professionals to facilitate the goal of person-centred care.

The routine use of patient‐reported outcome measures can contribute to the goal of person‐centred HIV care.

The ‘Positive Outcomes’ measure reflects what matters to people living with HIV and is shown to meet the requirements of a valid, reliable and responsive measure for use in clinical practice.

Findings demonstrate that the measure addresses the core domains of importance to people living with HIV: ‘Emotional wellbeing’ ‘Interpersonal and sexual wellbeing’ ‘Socioeconomic wellbeing’ and ‘Physical wellbeing’.

INTRODUCTION

Despite advances in antiretroviral therapy (ART), people living with HIV have worse health‐related quality of life (HRQoL) than the general population [1]. This is due to their high burden of physical symptoms, poorer mental health and social and spiritual concerns [2, 3, 4, 5, 6].

Good psychosocial care and communication with HIV professionals are associated with improvements in clinical outcomes, adherence and retention in care [7, 8]. However, people living with HIV feel that routine clinical appointments do not always address the things that matter most to them, with implications for their engagement with, and outcomes from, treatment and care [9, 10].

Care that addresses the multidimensional concerns of people with HIV requires a person‐centred approach, which is a core principle of quality healthcare [11]. The World Health Organization (WHO) global strategy for people‐centred and integrated services recognizes that, particularly for long‐term conditions, care must respond to the individual's preferences and concerns, and be coordinated around their needs [12]. The UNAIDS global HIV strategy would be strengthened by the proposed ‘4th 90’, that is, optimizing HRQoL for people living with HIV [13].

Health systems must focus beyond viral suppression to integrated, person‐centred healthcare for people living with HIV [14]. Measurable improvement in patient‐reported outcomes is the endpoint of quality healthcare [15]. However, there has been little consideration of what these person‐centred outcomes should be, and how they could be measured and integrated into standard HIV care.

Person‐centred care incorporating patient‐reported outcome measures (PROMs) can improve quality of care, patient–clinician communication, clinical decision‐making and patient outcomes [16, 17]. Patient‐reported outcomes also predict viral rebound [4], all‐cause hospitalization [18] and survival [19]. At a service level, PROMs ensure that care is directed towards those outcomes that matter most to the population, thereby promoting quality and equity [20]. They also serve to test the effectiveness of person‐centred complex interventions [8] as end‐points in drug trials to ensure patient‐reported outcomes are not inferior for new treatments [21], and as a screening tool [22].

HIV community groups and professionals have advocated for person‐centredness as standard of HIV care [23] and for a PROM to be used within routine practice [24]. A recent review found that there is currently no ‘gold standard’ HIV PROM [25]. HIV‐specific PROMs have been developed for single dimensions (e.g. depression, stigma, disability) [26] and for the construct of HRQoL [27]. Some PROMs have been successfully implemented in routine HIV practice [28]. However, there is no single, brief, valid person‐centred multidimensional tool that reflects the concerns of people with HIV and is adequately specific, responsive and actionable to drive and evaluate routine care.

This paper reports findings from a collaborative research programme to improve the person‐centredness and quality of HIV care through the development and validation of a brief PROM called Positive Outcomes. The construct measured relates to symptoms and concerns of adults living with HIV, in line with the WHO definition of health, i.e. ‘physical, mental and social well‐being’ [29]. Each item measures the extent to which the symptom or concern has affected the respondent in the previous 4 weeks. The intended purpose of the tool is two‐fold: first, for use in routine clinical practice, enabling the person living with HIV and their clinicians to rapidly identify their most burdensome symptoms and concerns from a set of core outcomes that commonly affect this population; and second, for use as a valid outcome measure in research.

We previously used qualitative approaches to inform face and content validity of the tool and to determine end‐user views on format and implementation [30]. Subsequent data described the community and multi‐professional item generation process, and findings from cognitive interviews and refinement of the tool's 23 items, which include one open text item and one item of global well‐being [31] (see Appendix S1 for full measure). The aim of the present phase of the study was to assess the validity, reliability and responsiveness of Positive Outcomes. Specific objectives were to assess: (a) validity (structural validity, convergent and divergent validity, discriminant validity); (b) reliability (internal consistency, test‐retest reliability); and (c) responsiveness of the tool.

METHODS

Design

This cross‐national measurement study applied the Rothrock [32] and COSMIN [33, 34] methodological guidance for the development and testing of health measurement scales.

Recruitment

This study was performed as a sub‐study within the EmERGE programme [35, 36] which co‐designed, implemented and evaluated a digital health pathway for people living with stable HIV. The pathway was validated within a mixed‐methods prospective longitudinal multicentre study.

Individuals with HIV were enrolled from outpatient HIV clinics in five European cities (Antwerp, Barcelona, Brighton, Lisbon, Zagreb) and invited to complete Positive Outcomes from April 2018 until the end of the study in October 2019. Eligible participants – aged ≥ 18 years, with documented HIV infection, able to provide written, informed consent, in possession of a smartphone, tablet or similar technology supporting the mHealth platform, clinically stable on ART [defined as being on ART for at least 1 year; unchanged for at least 3 months; two undetectable viral load tests (VL < 50 copies/mL), no current pregnancy; without any new World Health Organization clinical stage 2, 3 or 4 events within 12 months] [37] were identified by clinicians at sites. At some sites, a data search was used to identify eligible patients, while at other sites eligible participants were identified sequentially in the clinic.

Data collection and management

At baseline, informed consent was received from eligible individuals who were then invited to download the EmERGE mHealth application and link securely to the clinic database via a platform within the hospital firewall. Questionnaires were completed at baseline and again at 12 months. Data collected and used in this analysis were as follows: Positive Outcomes (see Appendix S1, collected from April 2018); patient activation using the PAM‐13 [38] (identified in a systematic review as the most valid measure of patient empowerment) [39]; health‐related quality of life using the PROQOL‐HIV [40] (identified as the HrQoL measure with best relevance for people living with HIV) [27]; ‘successful ageing’ using the FRAIL questionnaire [41]; and virological outcomes.

Analysis

Questionnaire data were entered onto an electronic Case Report Form (eCRF) and analysed using Stata 16.1 [42]. All analyses were performed using available cases. Descriptive analysis was conducted for sample demographic and clinical characteristics. Positive Outcomes scoring was carried out as follows. We calculated the mean of completed item scores [excluding question 1 (open text response) and question 2 (general health over the past 4 weeks)] when at least 80% (17/21) of the remaining items (questions 3–23) had been completed. Questions were scored 0–5 and the overall average had the same range, where a higher score indicates greater worries/problems. PAM‐13 is totalled for the 13 items, each of which score 1–4, and then the total is transformed to 0 (worse score) to 100 (best score). The 43‐item PROQOL‐HIV (scored 0 = never to 4 = always) has eight domains: physical health and symptoms (nine items), treatment impact (10 items), emotional distress (four items), health concerns (four items), body change (four items), intimate relationships (three items), social relationships (two items) and stigma (two items). FRAIL includes five components – fatigue, resistance, ambulation, illness and loss of weight – each with a score range of 0–5 (0 = best to 5 = worst) and represents frail (3–5), pre‐frail (1–2) and robust (0) health status.

Validity

Structural validity

We conducted an exploratory factor analysis to identify important latent factors that comprise the broader tool. Cumulative variance explained, Kaiser's rule, a scree plot, parallel analysis and the interpretability of resulting factor structures were considered before deciding on the number of factors to retain [43]. Promax (oblique) factor rotation was used, allowing correlation between factors. Factors were interpreted by the team and named according to the construct measured collectively by the items in a given factor. Cross‐loading items were reviewed to determine in which factor they loaded most, and for any item that did not load to the factor structure we appraised its uniqueness.

Convergent validity

Following assessment of score distributions we calculated correlations between Positive Outcomes domains and overall score with PAM‐13 score and PROQOL‐HIV total score. Spearman's rank correlation (ρ) was used due to skewed score distributions. Following exploratory factor analysis (structural validity, as described earlier), we generated the following hypotheses for strong correlations between (a) Positive Outcomes factor 1 score and PROQOL‐HIV ‘emotional distress’ domain score; (b) strong correlation between Positive Outcomes factor 2 score and PROQOL‐HIV ‘intimate relationships’ domain score; and (c) Positive Outcomes factor 4 score and PROQOL‐HIV ‘physical health and symptoms’ domain score. We applied Evans’ criteria, i.e. Spearman's ρ < 0.20 is very weak, 0.20–0.39 is weak, 0.40–0.59 is moderate, 0.60–0.79 is strong and ≥ 0.80 is a very strong correlation [44]. Factor scores were calculated as the mean of the items in the factor, where at least 80% of factor items had been completed.

Discriminant validity

We compared average Positive Outcomes scores between known groups: robust versus pre‐frail/frail (FRAIL) and PAM‐13 level 3/4 versus PAM‐13 level 1/2. Two logistic regression models used Positive Outcomes average score as the independent variable and pre‐frail/frail group as the dependent variable for model 1, and PAM‐13 level 1/2 versus level 3/4 for model 2.

Reliability

Internal consistency reliability

We measured using Cronbach's α for the scale (excluding global item and open text item) and for each of the factors that resulted from the exploratory factor analysis, applying a less conservative α threshold of 0.6 for non‐redundant multidimensional measures [45].

Test‐retest reliability

We identified a group of participants with ‘consistent’ scores (defined as not changing) on the following variables between first and second completion of the Positive Outcomes questionnaire: PAM‐13 level (remaining within levels 1/2/3/4), and frailty status (remaining within binary category of robust/pre‐frail) and having an undetectable viral load test result.

Responsiveness

We used data from participants who completed the Positive Outcomes, PAM‐13, EQ‐5D‐5L and PROQOL‐HIV measures twice and reported a change in PROQOL‐HIV average domain score between time points. We determined the relationship between PROQOL‐HIV score change and Positive Outcomes score change using Pearson's correlation coefficient.

Ethical approval

The research was conducted in accordance with relevant confidentiality, ethical and legal considerations [46, 47]. Ethics approvals were obtained from the sponsor and each institution (The Ethical Committee for Clinical Research of the Hospital Clinic de HC‐IDIBAPS; the South East Coast ‐ BSUHT & Sussex Research Ethics Committee 16/LO/2122 10Jan17; the Institutional Review Board of the ITM; the Ethics Committee members of the University Hospital for Infectious Diseases ‘Dr Fran Mihaljevic’, UHID, Croatia; and the Ethics Committee for Health, Centro Hospitalar de Lisboa Central CPE).

RESULTS

Sample characteristics

Data from 1705 participants were available for this analysis; 1392 participants completed Positive Outcomes at one time point, and 313 completed it at two time points (i.e. 12 months apart; see Table 1).

TABLE 1

Completion of time points by site

	Brighton, UK	Barcelona, Spain	Antwerp, Belgium	Zagreb, Croatia	Lisbon, Portugal	Total
1 completion	358	387	226	234	187	1392
2 completions	102	124	47	40	0	313

The majority of the sample was male (92.4%), Caucasian (77.1%) and had a median age of 45.0 years (IQR: 38.0–52.0) (see Table 2).

TABLE 2

Sample baseline characteristics (N = 1392)

	n	Median (IQR) or %
Age (years)	1392	45.0 (38.0–52.0)
CD4 (cells/μL)	974	740.7 (577.0–925.0)
Number of comorbidities	1318	0 (0–1)
Undetectable viral load	1389	‐
Sex
Male	1286	92.4%
Female	106	7.6%
Ethnicity
Asian	14	1.0%
Black	59	4.2%
Caucasian	1073	77.1%
Hispanic	85	6.1%
Other	161	11.6%
Non‐national
Yes	1111	79.8%
No	281	20.2%
Injecting drug user
Yes	51	3.7%
No	1341	96.3%
Heterosexual
Yes	182	13.1%
No	1210	86.9%
Frailty
Robust	957	73.6%
Pre‐frail	317	24.4%
Frail	26	2.0%

Abbreviation: IQR, interquartile range.

Data completeness

Analysis of all items for time point 1 found a range of 96.5–97.9% completeness per item, with an overall average of 2.9% missing items. Overall average score (requiring at least 80% of items complete) could be calculated for 1354/1392 participants (97.7%).

Psychometric properties

After oblique rotation we found that one factor explained cumulative variance of 70%, while Kaiser's criterion (eigenvalue > 1.0) identified two factors in the structure. The scree plot of eigenvalues suggested a three‐factor structure (Figure 1). Parallel analysis suggested five factors, with a further four factors with eigenvalues greater than would be expected by chance (see Figure 1).

FIGURE 1

Scree plot of eigenvalues showing expected eigenvalues from parallel analysis

As there was no clear number of factors to retain based on the four methods used, which suggested between one and nine factors, factor structures retaining between two and five factors were examined to identify the most easily interpretable structure. The rotated factor loadings shown in Table 3 for factor loadings > 0.3 demonstrate that two items contribute substantially to more than one factor: ‘Worried about starting family’ (factors 2 and 3) and ‘Enough support’ (factors 1 and 3). The two items ‘Able to perform usual activities’ and ‘Enough information to manage my HIV’ did not contribute substantially to any factor. Four factors were found to be an interpretable structure and an acceptable compromise.

TABLE 3

Factor loading matrix of Positive Outcomes tool: oblique rotated factor loadings following Promax rotation showing factor loadings > 0.3 only

Item	Rotated factor loadings (< 0.3 blank)
	Factor 1	Factor 2	Factor 3	Factor 4	Uniqueness
3. Enough information to manage HIV					0.829
4. Affected by pain				0.670	0.545
5. Stomach problems				0.594	0.603
6. Memory problems				0.457	0.540
7. Trouble sleeping				0.502	0.569
8. Able to perform usual activities					0.748
9. Felt anxious	0.682				0.262
10. Felt depressed	0.788				0.235
11. Felt worried about sharing HIV status		0.360			0.748
12. Felt good	0.905				0.241
13. Felt at peace	0.849				0.270
14. Worried about safety in relationships		0.583			0.451
15. Worried about drug consumption		0.316			0.792
16. Worried about money			0.810		0.381
17. Worried about housing			0.880		0.319
18. Worried about immigration status			0.348		0.746
19. Enough support	0.384		0.395		0.561
20. Worried about sex or intimacy		0.824			0.293
21. Worried about sexual health		0.942			0.277
22. Worried about contraception		0.868			0.366
23. Worried about starting family		0.472	0.340		0.542

To simplify the factor structure, the following decisions were made regarding cross‐loading and non‐loading items (see final factor structure in Figure 2). ‘Worried about starting family’ was assigned to factor 2 as this is where it loaded most substantially (0.472, compared with 0.340 on factor 3). ‘Enough support from people around you’ was assigned to factor 1 as this is where it loaded most substantially (0.395, compared with 0.384 on factor 3). ‘Enough information to manage your HIV’ was allowed not to load on any factor but retained in the total score as it had the greatest uniqueness (0.829). ‘Able to perform usual activities’ was assigned to factor 4 as this is where it loaded most substantially (0.295, just below the arbitrary threshold of 0.300).

FIGURE 2

Four‐factor structure of the Positive Outcomes tool (n = 20 items)

We named the factors as follows: factor 1, ‘emotional wellbeing’; factor 2, ‘interpersonal and sexual wellbeing’; factor 3, ‘socioeconomic wellbeing’; and factor 4, ‘physical wellbeing’.

Divergent and convergent validity

There was strong evidence for a weak correlation between Positive Outcomes score and PAM‐13 score: ρ = −0.295 (n = 1295, p < 0.001). There was also evidence for a strong correlation between PROQOL‐HIV average domain score and Positive Outcomes score: ρ = −0.678 (n = 1247, p < 0.001). The magnitudes and directions of score correlations were consistent with our prior expectations. Higher scores for PAM‐13 and PROQOL‐HIV indicate higher participant activation/QoL, whereas a lower score for Positive Outcomes indicates a higher QoL. Hypotheses generated following exploratory factor analysis were supported as follows: Positive Outcomes factor 1 (emotional wellbeing) score and PROQOL‐HIV emotional distress domain score, ρ = −0.584, p < 0.001; Positive Outcomes factor 2 score (interpersonal and sexual wellbeing) and PROQOL‐HIV intimate relationships domain score, ρ = −0.481, p < 0.001; Positive Outcomes factor 4 score (physical wellbeing) and PROQOL‐HIV physical health and symptoms domain score, ρ = −0.618, p < 0.001.

Discriminative validity

Distribution of Positive Outcomes average scores was substantially different for both group comparisons, with the median Positive Outcomes average score changing in the expected direction (i.e. higher in the pre‐frail/frail and PAM‐13 level 1/2 groups; see Table 4).

TABLE 4

Known groups comparison of Positive Outcomes scores to PAM‐13 and frailty

	Robust			Pre‐frail/frail
	n	Median	IQR	n	Median	IQR
Overall average	955	0.57	0.33–0.90	342	1.10	0.67–1.55

Abbreviation: IQR, interquartile range.

A one‐unit change in the Positive Outcomes average score was associated with a 4.81‐fold increased odds of being in the less favourable frailty group (95% CI: 3.74–6.19, p < 0.001) and a 2.28‐fold increased odds of being in the less favourable PAM‐13 level group (95% CI: 1.79–2.90, p < 0.001).

The total measure and the factors all had high internal consistency exceeding the less conservative threshold of > 0.6 for multidimensional measures with non‐redundant items (see Table 5) [45].

TABLE 5

Internal consistency of Positive Outcomes

Item set (n = 20 items)	No. of items	Cronbach's α
Overall (questions 3–23)	21	0.872
Factor 1 – Emotional wellbeing	4	0.889
Factor 2 – Interpersonal and sexual wellbeing	7	0.777
Factor 3 – Socioeconomic wellbeing	4	0.646
Factor 4 – Physical wellbeing	5	0.669

For those who scored consistently on PAM‐13 and frailty measures between first and second completion (n = 115), the median Positive Outcomes change score was 0.00 (IQR: −0.19–0.24).

PROQOL‐HIV average domain score changed for 242/247 (98.0%) of participants with two completions; 220/247 (89.1%) changed average Positive Outcomes score; and 1/247 (0.4%) did not change PROQOL‐HIV average domain score or average Positive Outcomes score. There was evidence for a moderate negative correlation between the two (Pearson's correlation coefficient = −0.44, p < 0.001). As PROQOL‐HIV average domain score increases (improves), average Positive Outcomes score decreases (improves).

DISCUSSION

The Positive Outcomes questionnaire comprises four domains: factor 1, ‘emotional wellbeing’; factor 2, ‘interpersonal and sexual wellbeing’; factor 3, ‘socioeconomic wellbeing’; and factor 4, ‘physical wellbeing’. These collectively measure the construct ‘symptoms and concerns’ among adults living with HIV. Results of this study demonstrate that the Positive Outcomes measure has sound psychometric properties (validity, reliability and responsiveness) for measuring the construct ‘symptoms and concerns’ among adults living with HIV. The measure has benefited from adherence to methodological guidance in development and validation of health outcome measures, and from close involvement by the intended end‐users (i.e. people living with HIV, clinicians and commissioners).

In line with our development findings, the tool begins with the option for open responses to identify main concerns (see Appendix S1).

Content and face validity were previously established based on primary qualitative data and cognitive interviews [30, 31]. In relation to structural validity, the four‐factor structure reflects a multidimensional measure of core symptoms and concerns, within the limit of length of tool proposed by stakeholders in the development work. Only one item did not load onto any factor (‘Enough information to manage HIV’), and this item had a very high uniqueness value. Based on the prior face and content validity data, we do not interpret this as a redundant item.

With respect to convergent validity, Positive Outcomes is strongly correlated with the PROQOL‐HIV quality of life measure. Therefore, Positive Outcomes achieves our goal of measuring a related but different concept to quality of life. We also demonstrated convergent validity to PAM‐13 with respect to discriminative validity, i.e. increasing (worsening) Positive Outcomes score is associated with worsening frailty and worse PAM‐13 level. For internal consistency, although we had set a lower threshold of 0.6 due to the multidimensional, non‐redundant nature of the measure, α‐values were high.

With respect to responsiveness, improvement in PROQOL‐HIV score was correlated with improvement in Positive Outcomes score.

There are a number of limitations to this study. First, the EmERGE cohort comprises adults whose HIV is medically stable, i.e. clinically stable on ART with an undetectable HIV viral load and no other multi‐morbidities requiring frequent monitoring by HIV services. However, this does now largely reflect progress against the UNAIDS targets for people living in western Europe [48]. Conversely, the development and use of this measure are crucial in moving on our understanding of wellbeing from a restricted definition of ‘medically stable’ to a broader and person‐centred profile of the symptoms, limitations, psychosocial and spiritual concerns that impair health and function. Second, our sample is only of adults, and further work should be undertaken to understand and measure outcomes for children, who face specific and additional concerns [49]. With respect to age, we recruited a relatively young adult sample (IQR: 38–52 years old), and physical function may be a more important factor in older samples. Within our sample, the majority were Caucasian men who have sex with men. The sample for the initial stages of face and content validity was more heterogenous, and therefore we are confident that the items represent symptoms and concerns of people living with HIV. Routine implementation of the PROM with all clinic attendees could provide further evidence of validity and may widen the benefits of PROM use to all service users. Third, face and content validity were developed in two European countries, while the validation was completed in five countries. Therefore, we cannot presume face and content validity beyond the two original countries, although we did conduct a consultation with patient groups in each of the additional countries to appraise face and content validity and the items were endorsed.

CONCLUSIONS

There is now adequate data to move to implementation of Positive Outcomes. Stakeholders have identified a number of potential benefits of the tool in routine use [50]. For people living with HIV these include improved communication, assessment, empowerment and decision‐making. For clinicians, anticipated benefits are identification of ‘missed’ concerns, better referral and informing treatment decisions, improved monitoring change over time, informing service design and delivery, justification of spending and improved care provision. The data also identified a patient preference for different completion options, including electronic and paper versions. The evidence demonstrates that electronic and paper‐and‐pencil PROMs deliver equivalent measures [51]. Positive Outcomes followed best practice through inclusion of stakeholders from the earliest design stages onwards [52]. However, successful use in routine practice also requires careful implementation plans to achieve the potential benefits of PROMs [53], and it is currently unclear what the most successful approach might be for HIV clinics. As healthcare delivery moves to virtual models, the use of PROMS may enable clinicians to identify those who should be seen face to face. The development of systems to integrate tools, enable acceptable completion and data‐sharing systems, and support for data usage (e.g. decision support tools [54]) should now be a priority.

The Positive Outcomes measure presents an opportunity to achieve greater person‐centredness of care in line with the expected standards of HIV care [23]. It has sound psychometric properties, and strong community and clinical support. Focus should now turn to implementation in routine care, with evidence‐based strategies to achieve the potential of PROMs in HIV care.

CONFLICT OF INTEREST

The authors declare there are no conflicts of interest.

AUTHOR CONTRIBUTIONS

RH is the principal investigator for Positive Outcomes and led the design and overall scientific conduct. CIJ was lead statistician, with input to the analysis plan, decision‐making and interpretation, with contributions from SB, RS and KKO. KB led tool development and contributed to validation design. BW contributed patient perspective to all stages of design, interpretation and reporting. JW is EmERGE principal investigator and led the validation study within the consortium, contributing to design, analysis and reporting. The EmERGE consortium conducted the validation study within its programme and contributed to scientific design.

App S1

Click here for additional data file.