Outcomes of Echocardiography-Detected Rheumatic Heart Disease: Validating a Simplified Score in Cohorts From Different Countries.

Background The natural history of latent rheumatic heart disease (RHD) detected by echocardiography remains unclear. We aimed to assess the accuracy of a simplified score based on the 2012 World Heart Federation criteria in predicting mid-term RHD echocardiography outcomes in children from 4 different countries. Methods and Results Patient-level baseline and follow-up data of children with latent RHD from 4 countries (Australia, n=62; Brazil, n=197; Malawi, n=40; New Zealand, n=94) were combined. A simplified echocardiographic scoring system previously developed from Brazilian and Ugandan cohorts, consisting of 5 point-based variables with respective weights, was applied: mitral valveanterior leaflet thickening (weight=3), excessive leaflet
tip motion (3), regurgitation jet length ≥2 cm (6), aortic valve
focal thickening (4), and any regurgitation (5). Unfavorable outcome was defined as worsening diagnostic category, persistent definite RHD or development/worsening of valve regurgitation/stenosis. The score model was updated using methods for recalibration. 393 patients (314 borderline, 79 definite RHD) with median follow-up of 36 (interquartile range, 25-48) months were included. Median age was 14 (interquartile range, 11-16) years and secondary prophylaxis was prescribed to 16%. The echocardiographic score model applied to this external population showed significant association with unfavorable outcome (hazard ratio, 1.10; 95% CI, 1.04-1.16; P=0.001). Unfavorable outcome rates in low (≤5 points), intermediate (6-9), and high-risk (≥10) children at 3-year follow-up were 14.3%, 20.8%, and 38.5% respectively (P<0.001). The updated score model showed good performance in predicting unfavorable outcome. Conclusions The echocardiographic score model for predicting RHD outcome was updated and validated for different latent RHD populations. It has potential utility in the clinical and screening setting for risk stratification of latent RHD.

rheumatic heart disease

World Heart Federation

Clinical Perspective

What Is New?

A simplified echocardiography score developed from Brazilian and Ugandan cohorts, consisting of 5 point‐based variables, seem to predict unfavorable echocardiographic outcome in schoolchildren with latent rheumatic heart disease.

When applied to populations from 4 different countries, the simplified score accurately predicted unfavorable echocardiography outcome at 3 years (hazard ratio, 1.10; 95% CI, 1.04–1.16; P=0.001), with good discrimination of those at higher risk.

The area under the receiver operating characteristic curve (C‐statistic) was 0.70, 95% CI, 0.64–0.76.

What Are the Clinical Implications?

This simplified echocardiographic risk score has a potential application as a risk stratification tool following rheumatic heart disease screening in 4 distinct populations.

It can be an additional tool to guide clinical monitoring and secondary prophylaxis, especially in underserved areas.

More research is necessary for additional validation of the score in different settings and populations, to allow for its broad application.

Globally, rheumatic heart disease (RHD) is the most prevalent acquired cardiovascular disease in children and young adults, affecting 40.5 million people and causing 306 000 deaths annually. , RHD is strongly associated with poor socioeconomic conditions, resulting in an uneven global distribution. The highest burden of disease is seen in low‐ and middle‐income countries, where access to health resources is frequently suboptimal. In such settings, RHD diagnosis is frequently delayed so that at time of diagnosis the patient has established late sequelae and markedly symptomatic valve disease. RHD is a preventable disease and the role of primary prevention and secondary prophylaxis following a diagnosis of acute rheumatic fever is standard. Nonetheless, approaches for the management of latent RHD – RHD diagnosed by echocardiographic screening absent a history of acute rheumatic fever – remains controversial. ,

Latent RHD, is a relatively new diagnostic category. , , , Diagnostic definitions were initially heterogeneous. However, in 2012 the World Heart Federation (WHF) published standardised criteria for the diagnosis of latent RHD (ie, a diagnosis of RHD based on echocardiographic findings in patients that were asymptomatic without a known history of acute rheumatic fever). The WHF criteria define 3 diagnostic categories (definite RHD, borderline RHD, and normal), based on functional and morphological findings of the left‐sided valves.

Although diagnostic standardization enabled meaningful comparisons between RHD screening studies (a crucial step for research purposes) the prognosis of latent RHD and the impact of interventions such as secondary prophylaxis and intensive clinical follow‐up are still under investigation. Considering that disease progression, particularly from borderline to definite RHD, is considered a serious adverse health outcome, , , efforts have been made to identify subpopulations at highest risk of progression so as to target them with secondary prevention initiatives. For example, in Africa a subdivision of the definite category has been proposed based on the hazard for echocardiographic and clinical progression in those with different valvular manifestations of definite RHD. In addition, Nunes and colleagues developed a simplified echocardiographic score derived from screening studies in Brazil and Uganda which showed good discrimination of risk of progression in 2.3 years in a separate Ugandan population subset. Although follow‐up in this study was short‐term, the simplified echocardiography score holds promise as a tool for guiding patient care. However, a wider application of the score requires robust external validation in different settings.

In this study, we aimed to assess the accuracy of the simplified scoring system developed by Nunes et al, consisting of 5 components of the WHF criteria, in predicting mid‐term RHD echocardiographic outcomes in a cohort of children from 4 different countries who were previously diagnosed with subclinical RHD on screening echocardiography.

Methods

Upon reasonable request to the corresponding author, study data and analytical methods may be made available to other researchers for the purposes of reproducing the results or replicating the procedures of this study.

This is an external validation study using primary data from pre‐existing RHD screening cohorts that included systematic echocardiographic follow‐up of patients initially diagnosed with latent RHD. Lead investigators from different research groups were contacted and invited to participate in the pooled analysis. After acceptance, patient‐level baseline and follow‐up data of children (aged 5 to 18 years) diagnosed with latent RHD during screening studies conducted in 4 countries (Australia, n=62; Brazil [sample not included in the original derivation study], n=197; Malawi, n=40; New Zealand, n=94) were provided. The data incuded echocardiographic findings, derived from studies with portable or handheld devices, for each of the variables used in making a diagnosis under the WHF criteria. According to these criteria, borderline RHD is defined as a single type of finding (i.e., abnormal mitral morphology, or isolated pathological mitral/aortic regurgitation), while definite RHD includes combinations of these, or more advanced findings (i.e., abnormal mitral morphology plus pathological regurgitation, or mitral stenosis, or abnormal aortic morphology plus pathological regurgitation, or borderline findings at both valves).

Ethics approval for inclusion of each individual cohort data set into the pooled set was obtained from the local institutional review boards, and all patients included in this analysis signed an informed consent form before enrollment.

Data sets were merged into a single spreadsheet (Excel, Microsoft, Redmond, VA, USA), and data verification and standardization was performed by 2 investigators (B.R.N. and M.C.P.N.) before statistical analysis. If needed, original investigators were contacted by email to provide clarifications or additional/missing data. Baseline and follow‐up diagnoses by the readers from the original study teams were used; echocardiographic images were not centrally re‐analyzed for the purposes of this study, aiming to maintain the original observations.

The simplified echocardiographic score proposed by Nunes et al, consisting of 5 point‐based variables with respective weights (mitral valve anterior leaflet thickening [weight=3], excessive leaflet
tip motion [3], regurgitation jet length ≥2 cm [6], aortic valve
focal thickening [4], and any regurgitation [5]) was applied to the pooled population. In the original derivation study, the risk of unfavorable outcome of latent RHD over 3 years was estimated for 3 groups: low‐risk (0–6 points, 7% risk), intermediate risk (7–9 points, 30%) and high‐risk (≥10 points, 53%). Details of the derivation and validation methodology used to generate the simplified echocardiographic score have been described elsewhere. Summarily, 3 cohorts of children were used for: (1) score derivation (echocardiographic screening study in Minas Gerais, Brazil; n=9501), (2) score validation (echocardiographic screening study in Gulu, Uganda; n=7312), and (3) outcomes prediction (longitudinal echocardiography cohort in Kampala, Uganda; n=227). In derivation, variables independently associated with definite RHD were assigned points proportional to their regression coefficients.

An unfavorable outcome was defined as a worsening in diagnostic category (borderline to definite), remaining with definite RHD, a worsening in the grade of mitral or aortic regurgitation, or the development or worsening grade of mitral stenosis. A favorable outcome was defined as disease regression (ie, an improvement in diagnostic category), a reduction in the severity of valvular regurgitation, or remaining with stable borderline RHD (Figure S1).

Patient and Public Involvement

Patients and public were not involved in the design and conduct of this research.

Statistical Analysis

Statistical analysis was performed using SPSS® software version 23.0 for Mac OSX (SPSS Inc., Chicago, Illinois) and R for Statistical Computing version 4.0.3 (R Foundation, Vienna, Austria). As we used pre‐existing cohorts, no pre‐specified sample size calculation was performed, and we included all the schoolchildren enrolled in the 4 primary studies. Categorical variables, expressed as numbers and percentages, were compared between each cohort included in the pooled set and between the 3 outcome groups (progression from borderline to definite RHD versus remaining with definite RHD versus remaining borderline RHD or regression from definite to borderline RHD) using Fisher exact test. Continuous data, expressed as mean (±SD) or median (interquartile range [IQR]), were compared using the Student unpaired t‐test or the Mann‐Whitney U test, as appropriate. External validation was performed by applying the original echocardiographic score model to an independent population of schoolchildren with latent RHD to assess its discrimination and calibration in predicting mid‐term unfavorable outcome (Figure S1). Additionally, the score model was updated by recalibrating the intercept and slope , and its performance was assessed in the external validation data set using different methods. The overall model performance was assessed using the Brier score, calibration using the Hosmer‒Lemeshow test and calibration plots, and discrimination using C‐statistic. For validation of the updated model, a bootstrapping procedure with 200 samples was used. A separate analysis was performed excluding individuals under prophylaxis.

Kaplan‒Meier survival analysis was undertaken to assess unfavorable outcome ‐free survival (times‐to‐an unfavorable outcome, considering the follow‐up intervals) in the 3 risk categories defined by the original score (low‐risk [0–6 points]), intermediate risk (7–9 points), and high‐risk (≥10 points). Differences between these groups were assessed by the non‐parametric log‐rank test. A 2‐tailed significance level of 0.05 was considered statistically significant.

Results

At total, 393 patients (314 with borderline and 79 with definite RHD) were included in the final analysis, with median follow‐up of 36 (IQR, 25–48) months. Median age was 14 (IQR, 11–16) years and penicillin prophylaxis was prescribed to 16%. Detailed characteristics of the individual cohorts included in the pooled data set are presented in Table 1. Of note, the longest median follow‐up intervals were observed in New Zealand and Malawi (58 [IQR, 37–77] and 48 [IQR, 24–48] months, respectively). These countries also had the highest proportions of definite RHD cases at baseline (31.9% and 27.5%). The highest proportion of patients in the high‐risk group, according to the original score was observed in Australia. This group also had the highest rate of echocardiographic unfavorable outcome (Table 1). Overall, the number of patients in each risk strata was: low 182 (46.3%), intermediate 120 (30.5%), and high 91 (23.2%).

Table 1

Baseline Characteristics of Patients With Borderline and Definite Rheumatic Heart Disease

Variable	Australia (n=62)	Brazil (n=197)	Malawi (n=40)	New Zealand (n=164)
Screening setting	Remote communities	Public and private schools, primary care	Schools/surrounding communities	Urban and rural schools
Age range (for inclusion, y)	8–18	5–18	5–16	5–17
Screening personnel	Accredited echocardiographists	Physicians and non‐physicians	Pediatric cardiologist, ultrassonographer	Accredited echocardiographists
Echocardiography adjudication	1 expert pediatric cardiologist	2 independent cardiologists and 1 tie‐breaker	2 independent cardiologists and 1 tie‐breaker	Pediatric Cardiology Department
Echocardiography machines	GE Vivid e/i	GE Vivid Q, VScan	Phillips CX50	GE Vivid e
BPG recommendation	At the discretion of cardiologist	At the discretion of cardiologist	Definite cases	Definite or probable cases
Age (median, IQR, y)	14.0 (11.9–15.6)	14.0 (11.7–16.0)	11.1 (9.4–13.3)	N/A
Follow‐up time (mo, median, IQR, y)	44 (40–48)	31 (22–36)	48 (24–48)	58 (37–77)
Baseline diagnosis (n, %)
Borderline	51 (82.3)	170 (86.3)	29 (72.5)	64 (68.1)
Definite	11 (17.7)	27 (13.7)	11 (27.5)	30 (31.9)
Follow‐up diagnosis (n, %)
Normal	20 (32.3)	91 (46.2)	15 (37.5)	41 (43.6)
Borderline	23 (37.1)	54 (27.4)	17 (42.5)	35 (37.2)
Definite	19 (30.6)	42 (21.3)	8 (20.0)	17 (18.1)
Score categories (n, %)
Low	19 (30.6)	121 (61.4)	10 (25.0)	32 (34)
Intermediate	9 (14.5)	49 (24.9)	21 (52.5)	41 (43.6)
High	34 (54.8)	27 (13.7)	9 (22.5)	21 (22.3)
Unfavorable echocardiographic outcome* (n, %)	19 (30.6)	42 (21.3)	8 (20.0)	17 (18.1)

AR indicates aortic regurgitation; AV, aortic valve; BPG, benzathine penicillin G; GE, General Electric; IQR, interquartile range (Q1–Q3); MR, mitral regurgitation; MS, mitral stenosis; MV, mitral valve; N/A, not available; and RHD, rheumatic heart disease.

Unfavorable echocardiographic outcome: worsening in diagnostic category (borderline to definite), remaining with definite rheumatic heart disease, a worsening in the grade of mitral or aortic regurgitation, or the development or worsening grade of mitral stenosis.

Among children with borderline RHD, 42 (13%) progressed to definite, 114 (36%) remained stable, and 151 (48%) regressed to normal. For those with definite RHD, 44 (56%) remained definite, while 15 (19%) regressed to borderline, and 16 (20%) to normal (Table 1). Among 63 patients receiving penicillin prophylaxis, 22 (35%) had unfavorable outcome. The baseline echocardiographic findings for patients relating to variables used in the WHF criteria (stratified by favorable/unfavorable RHD outcome at follow‐up) are detailed in Table 2. Additional stratification according to progression/stabilization/regression is provided in Table S1.

Table 2

Number and Percent of Patients With and Without Each Echocardiographic Variable Who Presented Unfavorable Outcome at 2‐Year Follow‐Up

Valve	Variable (n=present)	Unfavorable outcome (n=86)
		Variable present (+)	Variable absent (−)
Mitral valve, n (%) *	Anterior leaflet thickening † (n=165)	50 (30.3)	36 (15.8)
	Chordal thickening (n=21)	13 (61.9)	73 (19.6)
	Restricted leaflet motion (n=46)	25 (54.3)	61 (17.6)
	Excessive leaflet tip motion (n=86)	31 (36.0)	55 (17.9)
	Mitral stenosis (n=3)	3 (100)	83 (21.3)
	Any regurgitation (n=338)	82 (24.2)	4 (7.3)
	Regurgitation seen in 2 views (n=156)	42 (26.9)	44 (18.6)
	Jet length ≥2 cm ‡ (n=298)	73 (24.5)	16 (16.8)
	Velocity ≥3 m/s for 1 envelope § (n=181)	49 (27.1)	37 (17.5)
	Pansystolic jet (color Doppler) (n=122)	23 (18.9)	63 (23.3)
Aortic valve, n (%) *	Irregular or focal thickening (n=17)	7 (41.2)	79 (21.0)
	Coaptation defect (n=4)	2 (50.0)	84 (21.6)
	Restricted leaflet motion (n=1)	1 (100)	85 (21.7)
	Leaflet prolapse (n=4)	2 (50)	84 (21.6)
	Any regurgitation (n=92)	17 (18.5)	69 (22.9)
	Regurgitation seen in 2 views (n=86)	16 (18.6)	70 (22.8)
	Jet length ≥1 cm ‡ (n=61)	12 (19.7)	74 (22.3)
	Velocity ≥3 m/s in early diastole § (n=48)	10 (20.8)	76 (22.0)
	Pandiastolic jet (color Doppler) (n=22)	3 (13.6)	83 (22.4)

Congenital mitral valve or aortic valve abnormalities were excluded.

Abnormal thickening of the anterior mitral valve leaflet ≥3 or >4 mm using harmonic imaging.

In at least 1 view.

Measurements available with the Vivid‐Q exams.

Validation of the Simplified Echocardiographic Score Model

The original echocardiographic score model applied to the independent population of schoolchildren showed poor calibration (Hosmer‒Lemeshow test, P=0.059), and the calibration plot showed that predicted values were lower than observed values (Figure S2). The discrimination was modest (C‐statistic of 0.67, 95% CI, 0.61–0.74). The score model with the updated intercept and calibration slope (with the coefficients intercept=−0.724 and slope=0.241) resulted in improvement in overall model performance (Hosmer‒Lemeshow test of 4.19, P=0.651) and the calibration plot improved considerably (Figure 1A). The discrimination also improved with C‐statistic of 0.70 (95% CI, 0.64–0.76) (Figure 2). The Brier score (0.156) indicated a good overall performance of the updated model. Bootstrap validation demonstrated that the model was well calibrated with the Hosmer‒Lemeshow test of 11.228 (P=0.129) and reasonable discriminaton (C‐statistic of 0.70 [95% CI, 0.59–0.80]).

Figure 1

Plotted predicted‐to‐observed adverse outcome for the (A) total sample and for (B) each risk score group in the pooled population after recalibration.

 

Figure 2

Receiver operator characteristic curve for echocardiographic score showing predicted probability of unfavorable echo outcome from the model (area under the curve=0.70). AUC indicates area under the curve.

 

Applying the updated echocardiographic score model (Table 3) to the validation population, the predicted unfavorable outcome rates in the low, intermediate, and high‐risk groups were 13.5%, 23.2%, and 36.9%, respectively. The observed rates in these groups were 14.3%, 20.8%, and 38.5% respectively (Figure 1B, Figure S3).

Table 3

Echocardiographic Variables Used for Score Aftere Recalibration of Intercept and Slope

Variable	β coefficient	SE	Z value	P value	Points
Mitral valve
Anterior leaflet thickening	2.941	0.597	4.922	<0.0001	3
Excessive leaflet tip motion	3.102	0.543	5.716	<0.0001	3
Regurgitation jet length ≥2 cm	5.601	0.705	7.941	<0.0001	6
Aortic valve
Irregular or focal thickening	4.460	0.970	4.597	<0.0001	4
Any regurgitation	4.794	0.718	6.679	<0.0001	5
Intercept	−10.221	0.893	−1.448	<0.001	…

The updated score model was significantly associated with unfavorable echocardiographic outcome in continuous format (hazard ratio, 1.101; 95% CI, 1.042–1.164; P=0.001, for each point increase). Survival rates (free of unfavorable outcomes) in low‐risk children at 1, 2, and 3 years of follow‐up were 96%, 89%, and 71%, respectively, compared with 92%, 88%, and 77% in the intermediate group and 94%, 87%, and 60% in the high‐risk group (log‐rank test, P=0.028, Figure 3). The high‐risk group was associated with unfavorable outcomes (hazard ratio, 1.851; 95% CI, 1.108–3.094; P=0.019), whereas no differences was found comparing intermediate and low risk (hazard ratio, 1.062; 95% CI, 0.611–1.846; P=0.831). The Kaplan‒Meier curves revealed a significant gradual decrease in progression‐free rates after 3‐year follow‐up in the high‐risk compared with low and intermediate groups (Figure 3).

Figure 3

Cumulative survival free of unfavorable outcome in children with echocardiography‐detected rheumatic heart disease according to 3 risk categories: low‐risk (0–6 points), intermediate risk (7–9 points), and high‐risk (≥10 points) of the simplified score, with results of the Cox proportional analysis.

 

Excluding children under secondary prophylaxis, the score’s association with unfavorable outcome remained unchanged: hazard ratio of 1.100 (95% CI, 1.031–1.172; P=0.004), with a slightly lower C‐statistic of 0.66 (95% CI, 0.58–0.73).

Discussion

Our study, using pooled patient‐level data from RHD screening cohorts originating in 4 different countries, revealed that a simplified risk score previously developed by Nunes at al. based on the WHF criteria , accurately predicted the risk of an echocardiographic unfavorable outcome in individuals previously diagnosed with latent RHD. The model was updated considering differences in disease progression and prevalence among the countries. Our results indicate that the model of Nunes et al may be a potential tool for guiding medical care of children diagnosed with latent RHD following screening programs.

In the decade since the publication of the WHF criteria, which standardized echocardiographic‐diagnosis of latent RHD, the production of large data sets from across the globe using these criteria have made it possible to assess the performance and weight of its individual components. The initial derivation and validation of a risk score by Nunes and colleagues demonstrated that discrimination between borderline and definite echo‐detected RHD in the derivation cohort was near‐perfect, and that in an external validation cohort from Africa the score was consistently able to predict disease outcome. One limitation of this study, however, was that a similar group of experts analyzed all the echocardiographic images collected for participants, which could have favored a more homogeneous interpretation. Subsequently, an external validation was performed in a Brazilian cohort, with good – although more modest – discrimination of those at risk of echo progression. Now, for the first time, the use of multinational longitudinal data, managed by different research groups, has allowed for a more robust assessment of the power of the score to predict outcomes.

Within the context of our study, it is important to evaluate the history and intent of the 2012 WHF consensus guideline : the criteria were developed to enable international standardization of the echocardiographic diagnosis of subclinical RHD, thereby allowing for a direct between‐study comparison on prevalence and severity of RHD in different settings. However, there are a number of recognized limitations about the utility and applicability of the variables comprising the WHF criteria as a single diagnostic tool. In particular, there are issues on the utility of the WHF criteria in lower‐resourced screening environments where echocardiography providers usually undergo limited training such that they may not be able to capture all images required to measure all variables included in the WHF criteria. In addition, the WHF criteria lack adequate prognostic value, noticeably for clinical outcomes associated with the natural history of RHD. , Furthermore, application of the complete WHF criteria requires fully functional echocardiography devices with spectral Doppler and such devices are frequently not available in settings where RHD is endemic. Considerable research efforts have been made to address these limitations, leading to the development of simplified screening protocols with diagnosis based solely on valvular regurgitation and single‐view acquisition. , , However practical these approaches may be, they exclude morphological abnormalities that may present early in the natural history of RHD, even if there is no existant valvular regurgitation. Aiming to improve the practical utility of the WHF criteria, some investigators regrouped the original set of variables, according to risk. For example, Beaton and colleagues stratified the original definite RHD category (which included prominent functional and morphological findings) into moderate and severe definite RHD, and demonstrated that these subcategories were prospectively associated with higher risk of progression.

The simplified approach proposed in our study addresses some of these inherent limitations of the WHF criteria. The 2 functional and 3 morphological components included in the model are easily identified by the substantially less expensive handheld echocardiography machines, even in the absence of spectral Doppler. Furthermore, it proposes a systematization of the risk subsets proposed by Beaton et al. Finally, the reduced set of diagnostic variables also favor the point‐of‐care identification of individuals with latent RHD by non‐cardiologists with limited training. While for the overall set of criteria studies have reported intrareviewer agreement of 89% and between 71.4% and 94.1% in Brazil and Uganda, respectively, and inter‐rater agreement of 92% and from 66.7% to 82.8%, , when handheld devices and simplified criteria are used, diagnostic agreement between experts and trained non‐experts is still considerable. , Thus, the risk score classification proposed in this study has the potential to be even more reproducible.

Our pooled data highlight the differences in nature and severity of RHD between screening cohorts worldwide. There were major differences in baseline characteristics between cohorts, such as the higher proportions of definite cases in New Zealand and Malawi, while the cohort from Australia demonstrated worse risk and progression profiles in sharp contrast to the milder phenotypes observed in the Brazilian RHD cohort. There are, in addition, known epidemiological differences as well as screening approaches in these different settings. In Oceania, for example, there is a disproportionally high prevalence of RHD with a trend to severe phenotypes in indigenous and rural populations where screening programs are focused. This contrasts with the more general school and primary care‐based screening approaches, guided by socioeconomic indexes, in Brazil and Malawi. , , This heterogeneity possibly led to a lower predictive accuracy of the echocardiographic score model when tested in new schoolchildren compared with that found in the development study, indicating the need for updating the model to adjust for the setting of the validation sample. ,

External Validation of the Echocardiographic Score Model

There are no generally accepted rules of how external validation should be perfrmed, but it requires documentation of predictors and outcome values in new individuals. Model validation is not simply repeating the analytical steps applied in the development study in other individuals to assess whether the same predictors and weights are found. The purpose of validation is to combine the information captured in the original model with information from new individuals, which likely have improved transportability to other individuals. ,

In the present study, we aimed to perform a geographical validation of a previously developed model in other countries, and also in Brazil, including a population of schoolchildren that was not involved in the development study. Therefore, we applied the original model with its predictors and assigned weights (eg, regression coefficients), as estimated in the original study. Subsequently, we measured the predictors and outcome values in the new sample, and the model’s predictive performance was quantified. As expected, we found a low predictive accuracy of the score model when tested in this population, what is likely to occur in geographical validations, in which a more stringent form of validation is used. Then, the model was updated to adjust to the local circumstances of the validation sample (eg, environmental exposure, genetics, age at diagnosis, ethnicity, socioeconomic backgrounds, health system organization). After updating, the model's performance improved as measured by its discrimination and calibration. Therefore, rather than developing another prediction model, our aim was to evaluate whether the previously developed score predicts latent RHD outcome in distinct populations. A much better alternative to redeveloping new models for each new patient sample is to update existing prediction models and adjust or recalibrate them to local circumstances or setting of the validation sample at hand. It is noteworthy that when a lower predictive accuracy is found, investigators tend to simply reject that model and develop or fit a new one, sometimes by completely repeating the entire selection of predictors. This leads to a loss of previous scientific information captured in the development study, which is counterintuitive to the notion that inferences and guidelines to enhance evidence‐based medicine should be based on as much information as possible. , ,

Several methods for updating prediction models have been proposed and evaluated. Recalibration methods are attractive because of their stability, which is related to the fact that few parameters are estimated. In our study, the intercept and the overall calibration slope were recalibrated, considering differences in RHD outcomes frequency between samples, and potential overfitting of regression coefficients in the original model. We used the simple updating methods with further improvement in the calibration of the model.

Progression of Latent RHD

Our findings reinforce the variable evolution of latent RHD. In all our cohorts, over one third of cases regressed to normal, mostly comprising those with borderline disease. This was slightly lower than rates of regression reported in previous global reports (47%–67%). , Within those individuals with borderline RHD, rates of progression to definite were overall low (13%) and this finding was impacted by the largest sample size from New Zealand. This finding is comparable with rates reported in a recent meta‐analysis. Given these outcomes, in the absence of morphological findings or aortic involvement, the category of borderline RHD may be cautiously evaluated as an overlap with “normal” echocardiography. In contrast, for individuals with definite RHD, over one half remained in this category, in accordance with published pooled data, , suggesting that this is a more clinically relevant category, as the persistence of associated morphological and functional abnormalities in 2 observations may be a surrogate for a continuing pathological process. For this reason, we considered remaining definite as an unfavorable echocardiographic outcome and, as previously suggested, , , higher risk scores were strongly associated with persistence over time.

It is important to note that we were unable to determine whether there was any association between use of secondary prophylaxis and progression or stability of disease as only a small proportion of our pooled cohort was prescribed prophylaxis (according to different local guidelines), and adherence was not systematicaly captured in some studies. For example, in Malawi, penicillin was recommended for all definite cases, as it is in Australian RHD guidelines, whereas in Brazil it was left to the discretion of the attending cardiologist. Given the low rates of secondary prophylaxis, our outcomes can be practically interpreted as being representive of the “natural” evolution of latent RHD. Currently, a large randomized clinical trial is underway in Uganda examining the impact of penicillin prophylaxis in children and adolescents with screen‐detected RHD (Gwoko Adunu pa Lutino; ClinicalTrials.gov no. NCT03346525). This study should bring definite answers in relationship to the association between secondary prophylaxis and progression of latent RHD.

Our model was better refined to predict echocardiographic outcomes in the highest risk category. In contrast to the original outcomes‐prediction study, for our cohort there was overlap between the low and intermediate risk categories on the Kaplan‒Meier curve. While this finding limits refinement, and may possibly arise because of the data heterogeneity, the ability of our model to identify the subgroup at highest risk of unfavorable outcome points towards its usefulness as a public health tool, and urges further investigation. In addition, a simpler and more accurate prediction of disease progression may improve the cost‐effectiveness of population screening, as the prevalence of latent RHD detected in echocardiographic studies is >7 times higher than that detected by auscultation, even though a smaller number, with high‐risk profile, will benefit from closer clinical monitoring and, presumably, prophylaxis to halt disease progression. Thus, accurate risk stratification at time of diagnosis potentially implies not only on redefinition of latent RHD epidemiology, but also on the approach of children and families.

Limitations

Our study has several limitations. First, follow‐up was relatively short to determine clinical outcomes, while advanced sequelae of RHD may take decades to develop. Thus, the score must be further assessed as longer follow‐ups are available. However, the investigation of early echocardiographic predictors of disease progression is valuable for risk stratification and planning of clinical follow‐up. Second, we used screening diagnoses determined by the original echocardiography readers, without reassessment of images. Although all studies that contributed to our pooled data applied the WHF criteria, inter‐reader variability may account for some heterogeneity of our data, especially in relationship to specific morphological findings. Nevertheless, we aimed to assess the application of the score in real life programs, in which central reads are rarely available. Third, imaging acquisition strategies differed among the different included studies. For example, some used handheld echocardiographic devices while others used standard portable devices. Furthermore, in some studies baseline scans were performed by experts while in others they were performed by non‐physicians. For this reason, the original score consisted of variables that could be measured without spectral Doppler and, again here, task‐shifting is a frequent approach in real‐life screening studies. Fourth, the original intercept and slope for the risk model required recalibration, possibly because of the above‐mentioned interstudy heterogeneity. This suggests that a fine tuning of the model may be necessary for specific populations and regions, as it is not possible to infer whether the score would be equally effective in different countries. Fifth, no clinical variables – such as sex or age – were incorporated into the model, once we opted to validate the original score, composed exclusively by echocardiographic findings. Lastly, no inferences about the impact of penicillin on subclinical RHD can be drawn from our data. Despite these limitations, to the best to our knowledge, this is the only study evaluating this novel simplified score in one of the largest cohorts of subclinical RHD. A good accuracy to identify a high‐risk subgroup suggests that the score may be a useful stratification tool with potential impact of patient management on secondary prophylaxis.

Conclusions

The simple risk score developed by Nunes and colleagues provided an accurate prediction of RHD status at 3‐year follow‐up in a pooled multinational cohort. The updated score showed good performance in different screening populations worldwide, and has potential value for risk stratification of latent RHD.

Appendix

PROVAR (Programa de Rastreamento da Valvopatia Reumática) investigators:

Alison T. Reese, Andrea Z. Beaton, Antonio Luiz P. Ribeiro, Breno D. F. Rezende, Bruno R. Nascimento, Clara L. Fraga, Craig Sable, Domingos Sávio G. Ferreira‐Filho, Frederico V. B. Macedo, Juliane Franco, Kaciane K. B. Oliveira, Luiza P. A. Santos, Márcia M. Barbosa, Maria Carmo P. Nunes, Rodrigo T. L. Rocha, Sanny Cristina C. Faria.

Sources of Funding

The PROVAR+ investigators would like to thank Edwards Lifesciences Foundation® for supporting and funding the primary care screening program (PROVAR+) in Brazil, General Electric Healthcare® for providing echocardiography equipment and WiRed Health Resources for providing online curriculum on heart disease and echocardiography. The Telehealth Network of Minas Gerais was funded by the State Government of Minas Gerais, by its Health Department (Secretaria de Estado da Saúde de Minas Gerais) and FAPEMIG (Fundação de Amparo à Pesquisa de Minas Gerais), and by the Brazilian Government, including the Health Ministry and the Science and Technology Ministry and its research and innovation agencies, CNPq (Conselho Nacional de Desenvolvimento Científico e Tecnológico) e FINEP (Financiadora de Estudos e Projetos). Dr Ribeiro was supported in part by CNPq (Bolsa de produtividade em pesquisa, 310679/2016‐8) and by FAPEMIG (Programa Pesquisador Mineiro, PPM‐00428‐17). Dr Nascimento was supported in part by CNPq (Bolsa de produtividade em pesquisa, 312382/2019‐7), by the Edwards Lifesciences Foundation (Every Heartbeat Matters Program 2020) and by FAPEMIG (grant APQ‐000627‐20). Medical students received scholarships from the National Institute of Science and Technology for Health Technology Assessment (IATS, project: 465518/2014‐1).

Disclosures

None.

Table S1

Figures S1–S3

Click here for additional data file.