Utility of the Infantile Hemangioma Referral Score (IHReS) as a decision-making tool for referral to treatment.

Background: The general paediatricians and primary care physicians sometimes face immense difficulty in referral judgements regarding which infantile hemangiomas (IHs) require referrals and when is the appropriate time to refer IHs for treatment. This resulted in the treatment being delayed beyond IHs' critical timeframe. The Infantile Hemangioma Referral Score (IHReS) has been recently developed, with the aim to solve this problem.
Objectives: The objective of the present study is to evaluate the reliability of IHReS and to assess the possibility of using this instrument in our country where a similar problem of delaying treatment of IHs is currently existing.
Methods: The present study was a prospective, cross-sectional study. Thirteen selected clinical cases were used to assess the reliability of IHReS among physicians who may have had the chance to deal with patients with IHs. The target physicians across the country were asked to participate in the study via an online platform (Google Forms) to decide whether to refer patients with IHs for treatment or observe. There were 3 steps of assessment: step 1, usual practice evaluation; step 2, using IHReS; step 3, retesting by using IHReS.
Results: Substantial agreement was observed after using IHReS (step 2) for interrater reliability, with Fleiss' Kappa values of 0.80 and 0.78 among IH experts and non-expert physicians, respectively. Regarding repeatability, in the test-retest assessments, Cohen's Kappa coefficient values revealed almost perfect agreement in intrarater repeatability for both experts and non-expert physicians (1.00).
Conclusion: IHReS is a simple, easy-to-assess tool for non-expert physicians. The benefit in the increase of interrater agreement was found in both IH experts and non-expert physicians. It has had the reliability to be used in making referral decisions regarding patients with IH for treatment among Thai physicians. Using IHReS can improve clinical outcomes by identifying which patient needs early intervention to minimise the possible complications. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Infantile hemangioma (IH) is a disease with a window of opportunity in which physicians can make timely intervention and prevent poorer outcome. This critical time frame for optimising outcomes can be missed if there are delays in referral or treatment.

The heterogeneous presentation of IHs poses a clinical challenge for physicians in determining the need for treatment and subspecialty referral.

The Infantile Hemangioma Referral Score (IHReS) is a simple, easy-to-assess tool that has reliability to be used to make decisions regarding referral of patients with IHs for treatment in both IH experts and non-expert Thai physicians.

Using IHReS can improve clinical outcomes by identifying the patients who need early intervention to minimise the possibility of complications.

Introduction

Infantile hemangioma (IH) is a disease with a window of opportunity that allows timely intervention and prevents poorer outcomes.1 This critical time frame for optimising outcomes can be missed if there are delays in referral or treatment. A judgement of whether to refer for treatment or observe IHs is sometimes a difficult decision especially among non-expert physicians. This is due to the unique characteristic of IH that has its own spontaneous regression over a period of time2; thus, most non-expert primary care physicians usually provide a main leading advice for those patients with IHs to be observed without intervention or treatment. However, some IHs became problematic ones later when they start to have a rapid progression during proliferative phase. Most primary care physicians may not be able to identify problematic IHs at the time of examination that resulted in the treatment delays.

A similar problem of delayed referral of IHs for treatment is also in occurrence in our country. Most of the general paediatricians and primary care physicians face a difficulty in referral judgement to decide which IHs and when is the appropriate time to refer IHs for treatment to avoid the consequence of delayed treatment beyond the critical time frame.

Léauté-Labrèze et al recently proposed the Infantile Hemangioma Referral Score (IHReS) as an initial tool for primary care physicians to make their decisions to refer patients to expert centres.3 This tool was developed by the experts from seven different countries across the European countries. It had a high sensitivity of 96.9% which is suited for screening purposes. After IHReS has been published, we all agree that this may be a useful instrument to solve the problem of delayed treatment among patients with IHs. Therefore, this became the objective of the present study to evaluate the reliability of IHReS and to assess the possibility of using this instrument among Thai physicians.

Methods

Data collection

This was a prospective, cross-sectional study conducted in Thailand. Thirteen selected clinical cases were used to assess the reliability of IHReS among physicians who may have had the chance to deal with patients with IHs. The target population—paediatric dermatologists, general paediatricians and primary care physicians across the country—were asked to participate in the study via an online platform (Google Forms). Individual participants gave consent to the study by replying back the online questionnaire.

The participants were asked to make a decision whether to refer for treatment or observe individual 13 selected clinical cases provided with a clear high-quality image with essential history and physical examination. Three steps of the study intervention were designed: step 1, usual practice assessment of the selected clinical cases without reference to the IHReS; step 2, completion of the IHReS questionnaires of the same selected clinical cases; step 3, completion of the IHReS questionnaires for a second time (test–retest) 1 week after. The authors attached IHReS together with selected clinical cases via the Google Forms; thus, all participants were able to make a decision and submit their answer in one step. We used a personal code that was created individually by each participant to match the answer in step 3 which were made a week later, with the previous answers in steps 1 and 2.

The number of the target population in the study was calculated from the determination of sample size for estimating proportions with expected agreement of 0.8 with a margin of error of 0.1. For a confidence level of 95%, α was set at 0.05 and the critical value was 1.96. This resulted in a total calculated participant requirement of at least 62 participants.

Statistical methods

At the end of the study, the collected data were analysed using STATA software V.10 (StataCorp LP). Descriptive statistical methods—means, SDs, medians and frequencies—were used to analyse the demographic data. Internal consistency was calculated by using Cronbach’s alpha. Fleiss’ Kappa was used to test interrater agreement, while Cohen’s Kappa coefficient was used to analyse agreement of the repeatability decisions (intrarater agreement). Statistical significance was set at p value <0.05.

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Results

A total of 94 questionnaires were sent out to the target population—paediatric dermatologists, general paediatricians and primary care physicians across the country—via online platform (Google Forms), and 68 were returned. There were 28 primary care physicians, 36 general paediatricians and 4 paediatric dermatologists who participated. The majority of participants (56 physicians) were experienced in treating patients with IHs, eight physicians have never had the experience in treating this condition and four paediatric dermatologists were IH experts. Sixty-four participants have not known IHReS before participating in the study.

There were 13 selected clinical cases of IHs in the present study. Internal consistency tested by Cronbach’s alpha revealed a value of 0.88. The participants were classified into two groups, the IH expert group (4 paediatric dermatologists) and the non-expert group (64 participants: 28 primary care physicians and 36 general paediatricians). Sixty-eight participants completed the steps 1 and 2 questionnaires. The decision made at step 1 (usual practice assessment without IHReS) revealed moderate agreement for interrater reliability in IH experts, while fair agreement was observed in non-expert physicians, Fleiss’ Kappa values=0.42 and 0.23, respectively (table 1).

Table 1

Steps of the study interventions and the agreement results in IH experts and non-expert physicians

	Step 1: Usual practice assessment	Step 2: Completion of the IHReS questionnaire	Step 3: Completion of the IHReS questionnaire a second time (test–retest)
Statistical measurement	Interrater agreement(Fleiss' Kappa)	Interrater agreement(Fleiss' Kappa)	Interrater agreement(Fleiss' Kappa)	Intrarater agreement(Cohen’s Kappa)
IH experts	0.42	0.80	0.87	1
Non-expert physicians	0.23	0.78	0.81	1

IHReS, Infantile Hemangioma Referral Score.

For both expert and non-expert physicians, there were substantial agreement for interrater reliability at step 2 (completion of the IHReS questionnaires); Fleiss’ Kappa values=0.80 and 0.78, and almost perfect agreement was observed for interrater reliability in both groups, with Fleiss’ Kappa values for step 3 (IHReS retesting) of 0.87 and 0.81, respectively. Table 1 shows the steps of the study interventions and the agreement results in IH experts and non-expert physicians.

Regarding repeatability, in the test–retest assessment, Cohen’s Kappa coefficient values revealed almost perfect agreement in intrarater repeatability for both IH experts and non-expert physicians (1.00).

The average time needed to complete IHReS per each case was 12.59 s (SD 3.55). A satisfaction survey was sent to all 68 participants. The survey consisted of four questions and the value of Cronbach’s alpha for the satisfaction was acceptable, α=0.72. Percentage of participants’ satisfactions in four different aspects of IHReS are represented in table 2.

Table 2

Participants' satisfaction of IHReS in different aspects

The participants' satisfaction of IHReS in different aspects	Number of participants (N)					Total
	Strongly agree	Somewhat agree	Neither agree nor disagree	Somewhat disagree	Strongly disagree
IHReS helps in making decision to refer patients with IHs for treatment	46	18	4	0	0	68
IHReS is an easy-to-use tool	40	20	6	2	0	68
IHReS shortens the duration in decision-making process	46	12	6	4	0	68
Physicians will use IHReS to make decisions to refer patients with IHs in the future	50	16	0	2	0	68
Total	182	66	16	8	0	272

IHReS, Infantile Hemangioma Referral Score.

Discussion

IHs are commonly encountered in primary care and most often remain asymptomatic, resolving without sequelae. Even though certain characteristics are associated with a greater risk of complications, associated anomalies and disfigurement, most non-expert physicians usually provide the main leading advice for those patients with IHs to be observed instead of early intervention or treatment. The updated consensus guidelines1 2 4–6 had provided a suggestion of early treatment and timely intervention; however, the heterogeneous presentation poses a clinical challenge for physicians in determining the need for treatment and subspecialty referral. The heterogeneous presentation included patient age,7 IH type, different sizes of IHs, numbers of IHs, characteristics, locations, anatomical patterns,8 revealing of complications, timing of the IHs’ growth and parental preferences. The choice of active non-intervention as the primary approach to uncomplicated lesions was usually made. Life-threatening and function-threatening IHs, as well as IHs associated with a high risk for disfigurement and scarring, necessitate systemic treatment. The major problem for non-expert primary care physicians is determining the appropriate time for treatment of each individual case. This problem had become more evident, thus, the development of many IH scoring systems which aimed to provide an objective and standard measurement for early detection of problematic IHs and as a follow-up tool during the treatment.

Scoring IHs is challenging because of the heterogeneity of their morphology, behaviour and response to treatment. Many IH scoring systems have been developed during the past decade. Each scoring system has its own advantages for a variety of purposes. The Hemangioma Activity Score was developed to measure proliferative activity of IHs.9 This instrument has been used to monitor IH responses during the treatment.9–11 The Hemangioma Severity Scale (HSS) and Hemangioma Dynamic Complication Scale were developed shortly after with an objective to measure severity of IHs and the complications of IHs for longitudinal usage.12 The Hemangioma Activity and Severity Index was recently developed with a purpose to combine the proliferative activity score together with the severity index in one unified scoring system.13

All mentioned instruments are valid and used to measure disease severity that are needed to substantiate the benefit of therapies for IHs.11 As a utility of triage purpose, the cut-off values of the HSS of 6 or lower and 11 or higher could be used as a triage tool for propranolol treatment.14 Another study also revealed the cut-off values of IHs with total HSS scores of 6 or greater should be referred for subspecialty evaluation.15 As a triage purpose, the HSS may be a useful tool for primary care physicians in identifying high-risk IHs that may benefit from therapy. The HSS is a one-page scale with scoring items that require thorough information to complete the total score. The process is somehow needed to be refined to get to the standard results.

Léauté-Labrèze et al recently developed IHReS as an initial tool for primary care physicians to make their decision to refer patients to expert centres.3 This is a two-step easy-to-use tool for non-expert physicians, provided with some drawing pictures indicating striking location and practical notice points in making decisions. This tool is free to use and is available to be downloaded from www.ihscoring.com (figure 1). After IHReS efficacies had been published, we all agree that this may be a useful instrument to solve the delayed treatment among patients with IHs. Thus, initiation of the present study was set to evaluate reliability of IHReS and to assess the possibility of using this instrument among Thai physicians.

Figure 1

Infantile Hemangioma Referral Score (IHReS). This tool is free to use and is available to be downloaded online (wwwihscoringcom).

Our study was conducted in Thailand among the target physicians who deal with patients with IHs in their real practices that include primary care physicians, general paediatricians and paediatric dermatologists. The present study revealed that non-expert physicians had fair interrater agreement (Fleiss’ Kappa=0.23) at step 1 (usual practice assessment), while moderate agreement was observed in IH experts, Fleiss’ Kappa=0.42. This finding reflected that a problem of timely decision-making in treating IHs occurred more often in the non-expert physicians by the usual assessment without the assisting instruments. However, interrater reliability increased to substantial agreement at step 2 (use of IHReS) in both groups. The result correlated to the findings in the validation study of IHReS in 20203 and also reflected that the use of assisting instrument (IHReS) can help physicians in making their decision to refer patients with IHs for treatment. Our findings revealed that there was an increase in mutual agreement and acceptance after using IHReS not only among non-expert physicians—the IH experts also had benefited by the use of this score with an increased interrater reliability; Fleiss’ Kappa in IHs experts were 0.42 at step 1 and 0.80 at step 2, respectively. The IHReS also provided a consistent result after retesting at 1 week later with almost perfect interrater and intrarater repeatability in both groups (table 1).

The study design that was done via the online platform made the authors concerned about the returned questionnaires’ compliance, thus we decided to limit the number of selected clinical cases in the present study to shorten the time to complete the questionnaires. Therefore, there might be some selection bias of some difficult or controversial clinical cases that affected the decision by usual assessment without IHReS. However, the findings of discriminate decisions between usual assessments versus using IHReS supported the evidence that the triage screening tool for IHs as a decision to refer for treatment is essential.

In summary, IHReS was a simple, easy-to-assess tool for non-expert physicians. However, the present study also revealed that this tool is beneficial for IH experts as well. It took a short duration less than a minute to complete the score and had the reliability to be used to make a decision to refer patients with IH for treatment among Thai physicians. Using IHReS can improve clinical outcomes by identifying which patient needs early intervention to minimise the possibility of complications.